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EXECUTIVE  SUMfAiV 


The  OONABC  Soft  Skills  Training  Conference  was  conducted  at  the  Air  DeCeiiMr 
School,  12-13  Decenber  1972.  The  conference  achieved  its  stated  purpose  ol 
fostering  an  extensive  interchange  among  OONARC  representatives  and  invited 
speakers/participants  regarding  approaches  to  the  system  engineering  of 
soft  skill  training.  Reconmendations  were  submitted  for  facilitating  the 
systems  engineering  of  these  skills. 

After  a  welcome  and  presentation  by  BG  Ernst  E.  Roberts,  Assistant  Com¬ 
mandant  of  the  Air  Defense  School,  the  mission  and  nature  of  the  conference 
were  discussed  in  an  opening  statement  by  Colonel  G.  £•  Handley,  Chairman 
of  the  Conference.  Dr.  Robert  F.  Hager,  followed  with  a  presentation  on 
"Goals  Analysis"  which  emphasized  utilization  of  a  systematic  approach  to 
determining  training  requirements  from  "soft  skilled"  Job  requirements  by 
converting  these  Job  requirements  to  goals  and  behavioral  objectives.  A  TV 
recording  of  this  presentation  is  being  prepared  for  distribution  on  request 
to  CONARC  Training  Aids  Agency,  Fort  Eustis,  Virginia  23604.  In  order  to 
secure  said  presentation  a  60  minute  video  cassette  tape  must  be  submitted. 

A  series  of  presentations  concerning  the  application  of  the  systems  approach 
to  soft  skill  course  design  were  made  by  civilian  and  military  educators. 

Dr.  Paul  Whitmore,  HumRRO,  defined  soft  skills  and  explained  the  use  of 
a  behavioral  model,  developed  in  Project  MOIKAN,  as  a  tool  for  analyzing 
soft  skills.  The  remainder  of  the  morning  session  was  devoted  to  the 
presentation  of  four  approaches  to  the  systems  engineering  of  soft  skills 
by  separate  OONARC  schools.  The  Chaplain  School  presented  an  approach 
which  was  unique  in  that  it  involved  the  entire  resident  faculty  in  systems 
engineering  the  Chaplain's  Officer  Advanced  Course,  and  provided  students 
organized  into  small  learning  groups  the  opportunity  to  modify  the  means 
of  achieving  the  objectives  established  by  faculty  committees.  The  WAC 
School  presented  a  philosophical  approach  which  considered  cybernetic 
principles  and  recognizing  process  as  an  important  element  in  course 
structure.  In  the  Ordnance  School  approach,  broad  tools  statements  were 
accepted  initially  in  soft  skill  areas  and  later  reduced  to  more  specific 
behaviors  resembling  skills  and  knowledges.  This  approach  tends  to  re¬ 
cognize  skills  and  knowledges  as  the  real  training  requirements  in  the 
truly  soft  skill  areas.  In  the  approach  developed  by  the  Infantry  School 
for  designing  eight  combat  HOS  courses,  the  connion  soft  skill  tasks  are 
broken  down  into  more  specific  sub-tasks  which  then  can  be  measured. 

Dr.  Gerald  Nadler  of  the  University  of  Wisconsin  described  a  systems  de¬ 
sign  model  in  which  "identification  of  functions"  was  the  primary  feature. 

After  the  approaches  were  presented,  conferees  were  organized  into  small 
working  groups  and  each  of  the  presenters  were  scheduled  to  appear  for 
twenty-five  minutes  in  each  of  the  five  discussion  groups  to  answer  any 
questions  and  discuss  the  approach  previously  presented.  The  groups  pre- 


pared  reports  of  findings  with  conclusions  and  recoinnendations.  The 
najority  of  these  groups  agreed  that  the  application  of  the  systems  ap¬ 
proach  to  soft  skill  training  is  feasible  but  presents  an  increased 
challenge  not  faced  in  hard  skilled  courses.  It  was  tne  concensus  of 
the  groups  that  the  use  of  the  terms  **Soft  Skill"  and  "Hard  Skill"  be 
deemphasised  or  discontinued.  The  need  for  additional  cos^rrtiensive  ex¬ 
amples  and  training  in  the  systems  process  were  also  suggested. 

On  the  second  day  of  the  conference  three  siisultaneous  workshops  were 
conducted  which  considered  areas  of:  Methods  and  Media;  Developments  in 
Soft  Skill  Training  and  Quality  Control.  Suneuiries  by  each  of  the  chair¬ 
men  follow. 

Selected  conferees  attended  a  working  session  which  considered  the  pro¬ 
gress  of  HumRBO  Project  SMARTT  on  14  December  1972  and  participated  in 
a  data  collection  exercise.  Project  SMABTT  is  concerned  with  criteria 
and  techniques  for  selection  of  methods  and  Mdia  for  instruction. 
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Work  Unit  MODMAN  is  concerned  with  developing  procedures  for 
wystems  engineering  of  ''soft-^skills/'  such  as»  command,  supervision, 
counseling,  and  leadership.  At  the  outset  we  wanted  to  find  out 
what  the  term  "soft-skills'*  meant  to  staff  members  of  the  CONARC 
schools,  since  they  are  our  clients.  There  seemed  to  be  several 
dimensions  on  which  one  could  define  a  "soft"  to  "hard*'  continuum  of 
skills.  We  developed  a  questionnaire,  employing  nine  (9)  dimensions 
and  asked  knowledgeable  representative  of  the  various  CONARC 
schools  to  judge  35  general  job  functions  on  these  nine  dimensions. 

The  35  job  functions  cut  across  a  majority  of  officer  jobs. 

We  sent  two  questionnaires  to  each  of  29  CONARC  schools.  There 
were  35  respondents  from  21  of  the  schools  for  a  72%  overall  return 
rate.  One  of  the  delinquent  schools.  The  Army  War  College,  did  not 
see  the  relevance  of  the  questionnaire  content  to  their  curricxilum. 

Reliability  of  Measurement 

Our  first  analysis  was  to  determine  by  inspection  the  usefulness  of 
the  nine  dimensions  as  measurement  devices.  We  were  forced  to 
eliminate  one  dimension,  "Assessibility  of  Criteria,"  because  most 
respondents  failed  to  agree  on  where  the  various  job  functions  lay  on 
that  dimension.  In  other  words,  its  distributions  of  scores  were 
generally  either  flat  or  bi-modal.  Thus,  eight  dimensions  survived 
this  test  of  consistency. 

Three  of  these  dimensions  were  intended  to  help  clarify  and 
distinguish  between  what  is  meant  by  "soft"  and  "hard"  skills.  The 
remaining  four  dimensions  were  intended  to  test  the  hypothesis  that 
many  of  you  were  having  difficulty  in  trying  to  systems  engineer  the 
listed  job  functions  or  similar  ones  because  of  inadequate  document 
descriptions.  Thus,  the  results  of  the  questionnaire  will  be  discussed 
in  two  sections. 

What  are  Soft  Skills? 

The  CONARC  regulation  on  systems  engineering  (CON  Reg  350-100-1) 
of  training  defines  "soft  skills"  as: 

.  .job  related  skills  involving  actions  affecting  primarily 
people  and  paper,  e.g. ,  inspecting  troops,  supervising 
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office  personnel,  conducting  studies,  preparing  mainte¬ 
nance  reports,  preparing  efficiency  reports,  designing 
bridge  structures  /£•  2£7. " 


This  definition  leaves  much  to  the  imagination.  The  three  dimensions 
on  the  questionnaire  were: 

(1)  Degree  of  interaction  with  a  machine, 

(2)  Degree  of  specificity  of  the  behavior,  action,  or  process 
to  be  performed,  and 

(3)  Typical  kind  of  on-the-job  situation. 

Table  1  (Appendix  A)  shows  the  median*  response  by  the  35  respondents 
to  each  of  the  35  common  officer  job  functions  on  each  of  the  three 
dimensions. 

In  addition,  the  judged  importance  of  each  job  function  is  included  in 
the  table. 

The  dimensions  and  the  categories  on  each  dimension  are  defined  as 
follows: 

A.  Importance.  Indicates  how  important  the  judges  felt  each 
job  function  is,  relevant  to  jobs  held  by  graduates  of  the  Officer 
Advanced  Course  (C-22).  Categories  on  this  scale  were:  Great, 

High,  Moderate,  Low,  and  Little. 

B.  Degree  of  Interaction  With  a  Machine  Required  by  the  Job 
Function.  At  one  extreme,  an  individual  constantly  operates  a  machine 
(such  as  a  radar  scope)  0£  fills  out  a  piece  of  paper  (such  as  a 
maintenance  form).  At  the  other  extreme,  a  job  incumbent  may 
manipulate  machines  (or  forms)  as  entities  but  does  not  actually  operate 
them  or  fill  them  out  per  se.  For  example,  a  commander  may  use  the 


*NOTE:  The  median  measure  of  central  tendency  is  preferred  here 
because  the  scales  used  were  ordinal;  equal  intervals  cannot  be 
assumed.  As  it  turned  out,  for  almost  all  scales,  the  mean  was  more 
moderate  and  the  mode  was  more  extreme  than  the  median,  indicating 
rather  skewed  distributions  of  scores. 
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mental  image  of  radar  scopes  or  maintenance  forms  in  solving  a 
p  oblem,  but  he  need  not  interact  with  them  directly.  (In  a  sense, 
the  term  **man«a8cendant**  refers  to  the  latter  extreme,  where  man 
is  in  control  or  dominates.  In  contrast,  **machine«ascendant**  refers 
to  jobs  where  machines  dominate. )  Categories  here  were:  Always, 
Some,  Moderate,  Little,  and  Never  (or  indirectly), 

C.  Degree  of  Specificity  of  the  Behavior  to  be  Performed. 

This  dimension  also  includes  actions  or  processes  to  be  performed. 

At  one  extreme,  they  can  be  explicitly  stated  and  their  application 
on-the-job  is  also  quite  specific.  For  example,  there  probably  is 
only  one  way  to  change  the  oil  in  a  given  vehicle.  At  the  other 
extreme,  behavior,  actions,  or  processes  are  either  implied  within 
some  context  or  application  on-'ihe-job  is  quite  generalized.  An 
exc  pie  might  be  the  requirement  that  the  commander  be  able  to 
motivate  or  lead  troops,  whenever  and  wherever  the  situation  calls 
for  it.  Categories  on  this  scale  were:  Very  Specific  (V  Spec), 

Specific,  Moderate,  General,  and  Very  General  (V  Gen). 

D.  Typical  Kind  of  On^the-Job  Situation.  This  dimension 
defines  a  continuum  of  on- the -job  situations  from  established  to 
emergent.  In  established  situations,  (a)  physical  and  social 
environmental  conditions  are  known,  and  (b)  the  consequences  of 
alternative  courses  of  action  are  known.  On  the  other  hand,  in 
emergent  situations,  (a)  not  all  physical  and  social  environmental 
conditions  have  been  determined,  and  (b)  the  consequences  of 
alternative  courses  of  action  are  not  always  known.  Usually,  job 
functions  in  emergent  situations  are  also  associated  with  a  large  amoimt 
of  uncertainty. 

Judges  rated  each  job  function  relevant  to  on-the-job  situations  as 
they  were  deemed  to  exist  for  job  encumbents.  Categories  were: 

Very  Established  (V  Estab),  Established,  Medial,  Emergent,  and 
Very  Emergent  (V  Emer). 

Most  of  the  job  functions  were  judged  to  be  of  Great  importance;  none 
were  thought  to  be  of  low  or  little  importance  to  the  C-22  graduate. 

Likewise,  most  of  the  job  functions  were  judged  to  Never  '"interact 
with  a  machine";  only  two  job  functions  were  thought  to  even  interact 
to  a  moderate  degree  (#29  and  #30). 
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Result*  on  the  "degree  of  specificity"  dimension  were  also  pre¬ 
dominantly  one-sided  (i.  e. ,  General  or  Very  Gen).  Only  three 
job  functions  were  judges  Specific  or  Very  Spec  (#3,  #31,  and  #35). 

Finally,  and  consistent  with  the  above  dimensions,  only  three  job 
functions  were  judged  to  be  Established  (#3,  #30,  and  #31). 

In  general,  distributions  of  scores  on  the  three  dimensions  were 
very  similar  across  all  35  job  functions. 

However,  several  job  functions  revealed  inconsistencies  among  the 
three  dimensions.  For  example,  #3,  "Interprets  and  Uses  a  Military 
Map"  was  purposely  included  in  the  set  of  job  functions  as  a  "hard" 
skill  that  made  use  of  paper.  Apparently,  most  of  the  35  judges 
felt  that  using  paper  in  this  way  was  not  the  same  as  interacting  with 
a  machine.  On  this  basis,  it  would  be  categorized  as  a  "soft-skill.  " 

On  the  other  two -dimensions  79%  of  the  respondents  felt  that  processes 
for  using  a  map  were  explicit  and  constituted  a  specific  application. 
Sixty  three  percent  felt  that  the  job  situations  for  using  maps  were 
known.  The  responses  on  these  two  categories  would  cause  it  to  be 
categorized  as  a  "hard-skill.  '*  Thus,  it  appears  that  including  "paper" 
in  a  definition  of  "soft"  skills  requires  an  explanation  of  how  that 
"paper"  is  used.  Does  the  user  fill  in  standardized  blanks  or  does  he 
use  "paper"  to  manipulate  ideas  or  objects  as  mental  images. 

A  tentative  definition  of  soft  skills  might  be  formulated  as  follows: 
Soft-skills  are  (1)  important  job-related  skills  (2)  which  involve 
little  or  no  interaction  with  machines  (including  standardized  because 
the  situation  or  context  contains  a  great  deal  of  uncertainty;  that  is, 
we  don't  know  much  about  the  physical  and  social  environments  in 
which  the  skill  >occurs  and  we  don't  know  much  about  the  consequences 
of  different  ways  of  accomplishing  the  job  function.  In  other  words, 
those  job  junctions  about  which  we  know  a  good  deal  are  hard  skills 
and  those  about  which  we  know  very  little  are  soft  skills. 

Adequacy  of  Docxanent  Descriptions. 

The  systems  engineering  process  described  in  the  CONARC  regulation 
relies  heavily  on  the  use  of  Army  documents.  Hence,  our  next  area 
of  concern  was  with  the  adequacy  of  these  documents  for  systems 
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engineering  of  aoft  skills.  Four  dimensions  on  the  questionnaire  were 
CO'  cerned  with  documentation  in  addition  to  "Importance,  "  as  follows: 


A.  Importance.  Indicates  how  important  the  judges  felt  each 
job  function  is,  relevant  to  jobs  held  by  graduates  of  the  Officer 
Advanced  Course  (C-22). 

B.  Document  Description  of  Behaviors  to  be  Performed.  Here 
judges  were  asked  to  indicate  how  well  existing  Army  documents 
(manuals,  pamphlets,  reports,  etc.)  describe  behaviors,  actions,  or 
processes  for  performing  each  of  the  given  job  functions.  If  documents 
describe  them  in  such  a  manner  that  an  individual  can  actually 
perform  the  function,  given  only  the  information  in  the  documents, 
then  one  extreme  of  the  dimension  was  met.  On  the  other  hand,  if 
documents  fail  to  describe  necessary  behavior,  action,  or  processes  at 
all,  then  the  other  extreme  of  the  dimension  was  met. 

Categories  here,  as  well  as  the  next  two  dimensions  below,  were: 

Very  Adequate  (V  Ad),  Adequate,  Moderate,  Inadequate,  and  Very 
Inadequate  (V  Inad). 

C.  Document  Description  of  Conditions  of  Performance.  Here 
existing  Army  documents  were  judged  as  how  well  they  describe  the 
critical  characteristics  of  the  conditions  or  situations  in  which  each 
job  function  will  be  or  is  performed.  One  extreme  was  represented  by 
documents  which  fail  to  describe  any  situations  or  conditions.  Where 
existing  documents  describe  any  situations  or  conditions  in  such  a 
manner  that  an  individual  can  develop  a  broad  range  of  realistic 
(simulated)  training  situations  or  conditions  for  problem  practice  or 
testing  purposes,  then  the  opposite  extreme  was  represented. 

D.  Document  Description  of  Standards  of  Performance.  This 
dimension  asked  for  an  estimation  as  to  how  well  existing  Army 
documents  describe  standards  of  performance  or  criteria  for 
assessing  on-the-job  effectiveness  of  the  given  job  function.  The 
extremes  are  similar  to  those  outlined  above. 

E.  Specification  of  Job  Requirements,  Systems  engineering  of 
training  requires  that  the  analyst  develop  aT^trial  inventory  of  meaning¬ 
ful  and  useful  tasks.  Here  judgments  were  made  concerning  the 
degree  of  difficulty  in  accomplishing  this  step  for  each  given  job 
function.  Categories  here  were  Very  Easy  (V  Easy),  Easy,  Moderate, 
Difficult,  and  Very  Difficult  (V  Diff). 
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A  quick  •canning  of  Table  Z  (Appendix  B)  readily  reveals  that  although 
most  of  these  job  functions  are  judged  to  be  qui  te  important,  the 
existing  Army  documents  do  not  provide  an  adequate  description  of  the 
behaviors,  conditions,  or  standards  associated  with  most  of  these 
job  functions.  If  the  information  about  these  job  functions  is  not  in  the 
Army  documents  and  if  it  is  not  known  to  staff  members  at  the  CONARC 
schools,  then  it  is  likely  that  it  is  also  not  known  to  job  incumbents 
in  the  field.  We  clearly  need  some  different  approaches  that  will 
provide  us  with  good  information  about  the  behaviors,  the  job  situations, 
the  conditions,  and  the  standards  for  these  kinds  of  job  functions. 


THE  BEHAVIORAL  MODEL  AS  A  TOOL  FOR  ANALYZING  "SOFT  SKILLS" 


Dr.  Paul  G.  Whitmore 

HumRRO  Resources  Research  Office 

P.  O.  Box  6057 

Fort  Bliss,  Texas  79916 
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The  Analysit  of  Purposes  and  Situations 

We  cannot  evaluate  a  worker’s  performance  until  we  know  the  pur¬ 
poses  of  his  job  and  the  situations  in  which  the  job  is  to  be  performed. 
And  we  cannot  train  others  to  do  this  job  until  we  know  what  move¬ 
ments  or  acts  will  achieve  the  purposes  of  the  job  in  the  situations 
in  which  it  is  to  be  performed.  The  object  of  systems  engineering 
is  to  identify  the  purposes  of  the  job  and  the  situations  in  which  these 
purposes  are  to  be  achieved^  the  cognitive  and  perceptual -motor 
acts  for  achieving  these  purposes,  and  the  selection  practices  and 
training  ^'speriences  which  will  best  produce  sufficient  numbers  of 
people  to  do  the  job  at  the  least  cost. 

Let's  begin  with  the  identification  of  purposes  and  situations.  Some¬ 
times  we  analyze  purposes  first  and  sometimes  we  analyze  situations 
first.  And  sometimes  we  go  from  one  to  the  other  and  back  again. 
Much  depends  upon  the  specific  nature  of  the  particular  beast  with 
which  we  are  dealing. 

Generally,  when  we  are  analyzing  jobs  in  a  closely  integrated,  multi- 
job  organization  with  overall  organizational  goals,  we  will  analyze 
purposes  first.  For  instance,  if  we  were  analyzing  a  job  or  jobs  in 
a  combat  infantry  platoon,  we  would  begin  by  identifying  the  general 
missions  which  the  platoon  may  be  required  to  perform.  Each 
mission  would  be  analyzed  by  means  of  rational  processes  into 
successively  more  detailed  subordinate  purposes  until  we  reached  a 
level  in  which  a  purpose  can  be  performed  by  a  single  individual. 

This  is  the  level  of  job  duties. 

However,  when  we  are  analyzing  jobs  that  are  not  integrated  into 
a  multi-job  organization,  we  may  analyze  situations  first,  rather 
than  purposes.  An  outstanding  example  of  this  type  of  analysis  has 
been  conducted  by  McKnight  — it  is  concerned  with  driving  a 
private  automobile.  In  their  words,  the  "first  step  in  the  process 
was  to  identify  those  aspects  of  the  (highway  transportation)  system 
that  were  capable  of  creating  situations  to  which  the  driver  must 
respond-for  example,  curves  in  the  road,  traffic  control  devices, 
cars  ahead,  snow,  rain,  and  driver  fatigue.  Over  1,000  specific 
behaviorally  relevant  system  characteristics  were  identified.  " 


McKnight,  A.  James  and  Bert  B.  Adams,  Driver  Education  Task 
Analysis;  Task  Analysis  Methods,  Technical  Report  72-13,  HumRRO, 
April  1972. 
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These  system  characteristics  were  then  taken  singly  and  in  combination 
wi  \  each  other  to  identify  potential  critical  situations  with  which  a 
driver  may  have  to  cope. 

In  order  to  identify  the  specific  purposes  and  situations  which 
constitute  a  given  job,  we  begin  by  analyzing  the  system  in  which 
that  job  occurs.  This  initial  systems  analysis  in  the  system  engineer¬ 
ing  process  is  concerned  with  insuring  the  comprehensiveness  and 
validity  of  the  task  inventory.  With  regard  to  the  identification  of 
system  characteristics,  McKnight  and  Adams  note: 

Inasmuch  as  this  activity  formed  the  foundation  for  later 
analysis,  particular  pains  were  taken  to  see  that  the 
analysis  was  very  broad,  since  to  overlook  any  relevant 
system  characteristics  could  result  in  the  omission  of 
potentially  critical  .  .  •  behaviors. 

They  began  both  the  situation  analysis  and  the  purpose  analysis  by 
identifying  a  restricted  number  of  broadly  defined  classes  of  character¬ 
istics  **which  shape  the  responses  the  individual  must  make  as  a 
driver;"  that  is,  "that  might  impose  behavioral  requirements  upon 
the  driver.  "  The  broad  classes  of  characteristics  from  which  the 
situation  analysis  commenced  consisted  of  driver  characteristics, 
vehicle  characteristics,  roadway  characteristics,  traffic  character¬ 
istics,  and  characteristics  of  the  external  environment.  These 
general  characteristics  guided  the  identification  of  specific  character¬ 
istics  gleaned  from  various  sources  of  information.  The  initial  list 
of  characteristics  was  arranged  into  a  logical  hierarchy  or  tree- 
structure,  or  outline.  This  hierarchy  was  examined  for  logical  gaps 
at  each  level.  The  gaps  were  filled  in  and  in  this  manner  the 
hierarchy  was  extended  so  as  to  be  as  comprehensive  as  possible. 

Parts  of  the  final  hierarchy  has  as  many  as  seven  levels.  These 
situation  characteristics  were  then  taken  singly  and  in  combination 
to  define  the  situations  with  which  the  subsequent  analysis  would 
be  concerned. 

McKnight  and  Adams  also  conducted  an  analysis  of  the  system's 
purposes  or  goals.  It  was  not  nearly  as  elegant  as  the  situation 
analysis,  but  it  was  equally  important.  As  they  defined  it,  the 
purpose  or  goal  of  the  highway  transportation  system  is  to  assure 
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"the  movement  of  passengers  and  material  from  one  place  to 
another  with  safety*  efficiency*  comfort*  and  responsibility.  " 

The  characteristics  of  the  goal  impose  general  behavioral  require¬ 
ments  on  drivers: 

1.  "Safety  requires  that  di-ivers  behave  in  a  way  that  will 
minimize  the  chances  of  injury  or  property  damage.  •  • " 

2.  "Efficiency  requires  that  drivers  avoid  interfering  with 
the  rapid  and  economical  flow  of  traffic. . . " 

3.  "Comfort  requires  that  drivers  operate  in  ways  that  will 

not  cause  discomfort  to  passengers*  other  drivers*  or  pedestrains.  •  " 

4.  "Responsibility  means  that  drivers  should  be  morally  and 
financially  responsible  for  the  consequences  of  their  acts.  . .  " 

Although  the  authors  do  not  directly  say  so*  it  seems  apparent  that 
these  general  goal  characteristics  must  have  been  used  to  identify 
specific  goals  or  purposes  in  each  situation  defined  by  the  situation 
analysis.  These  specific  goals  defined  what  the  effects  of  the  driver's 
behavior  ought  to  be  in  the  situation.  Then  the  analysis  probably 
proceeded  backwards  from  the  necessary  effects  of  the  behavior  to 
the  identification  of  the  behavior  itself.  This  analysis  was  based  on  the 
operational  characteristics  and  principles  of  automobiles. 

The  Analysis  of  a  "Soft  Skill" 

We  can  specify  what  a  machine  operator  ought  to  do  if  we  know  the 
situations  or  environments  in  which  he  works*  the  purposes  of  his 
work  in  these  situations*  and  the  theories  of  operation  of  the  machines 
he  uses.  Now  suppose  we  were  dealing  with  a  supervisor  or  a  leader 
or  a  counselor.  He  is  not  a  machine  operator;  rather*  he  is  a  people 
operator.  He  directs  the  activities  of  people  toward  the  accomplish* 
ment  of  specified  purposes  in  specified  situations.  However*  he  must 
perform  under  a  handicap  not  imposed  on  machine  operators:  namely* 
his  job  activities  have  been  only  vaguely  identified  and  have  not  been 
derived  from  an  unequivocal  theory  of  operation  for  people.  As  a 
result*  supervisors  tend  to  seek  out  or  develop  their  own  theory  of 
people  operation  from  which  they  derive  their  job  activities  or  else 
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the  /  accept  the  common  myths  about  the  operation  of  people  which 
are  promulgated  by  our  culture. 

The  selection  of  an  appropriate  theory  of  people  operation  is  a  task 
for  the  behavioral  scientist.  But  even  among  such  experts,  we  will 
find  disagreements  and  myths.  It  is  an  area  fraught  with  the  vagaries 
of  ill-defined  terms,  which  must  be  hacked  through  with  a  semantic 
machete.  I  personally  prefer  to  deal  with  leadership  and  motivation 
job  functions  in  terms  of  principles  of  behavior  modification,  some¬ 
times  also  referred  to  as  contingency  management,  behavior  manage¬ 
ment,  reinforcement  theory,  or  operant  conditioning.  I  recommend 
this  approach  for  a  number  of  reasons: 

1.  The  basic  principles  are  relatively  simple  and  easy  to  learn 
and,  yet,  have  application  to  a  great  variety  of  situations. 

2.  Like  syste  r  -  engineering,  it  is  a  behavioral  approach  which 
deals  with  specifically  defined  behaviors. 

3.  It  leads  to  the  development  of  precision  management  processes 
in  which  behav  ior  is  monitored  and  progress  is  audited  towards  clearly 
specified  goals.  This  process  provides  a  basis  for  a  continual  assess¬ 
ment  of  effectiveness  and  for  timely  and  flexible  corrective  action. 

4.  It  provides  effective  and  sophisticated  alternatives  to 
intimidation,  punishment,  and  permissiveness  as  general  motivational 
and  disciplinary  treatments. 

I'm  not  going  to  subject  you  to  a  treatise  on  behavior  modification 
at  this  time.  There  are  a  number  of  excellent  publications  on  the 
topic,  some  of  which  are  listed  on  your  handouts  (Appendix  C). 

Although  behavior  modification  principles  constitute  an  appropriate 
theory  of  people  operation  for  leadership  and  motivation  job  functions, 
other  human  job  fxinctions  will  involve  other  theories  of  people  operation. 
For  instance,  if  we  were  dealing  with  target  recognition,  we  would 
select  an  appropriate  visual  perception  theory.  If  we  were  dealing 
with  complex  job  decisions,  we  would  select  an  appropriate  decision¬ 
making  theory. 


Now  let's  look  at  "worker  motivation"  as  a  job  function.  We  begin  by 
analyzing  ^e  term  "worker  motivation"  into  the  kinds  of  behaviors 
exhibited  by  motivated  workers.  What  does  the  motivated  worker  do 
that  makes  him  different  from  the  unmotivated  worker  ?  1  have  listed 
some  suggested  operational  characteristics  of  a  "motivated  worker" 
in  your  handout.  Clearly,  a  "motivated  worker"  should  perform 
some  adequate  amount  of  work  that  leads  to  the  attainment  of  organi¬ 
zational  goals.  He  should  exhibit  a  low  frequency  of  interference  with 
other  workers.  He  shoxild  coordinate  his  work  with  others  in  keeping 
with  organizational  goals.  If  necessary,  he  should  help  others  to 
perform  their  work,  if  he  is  able  to  do  so.  He  shoiild  minimize  the 
effects  of  outside  disruptions  on  his  own  work  and  that  of  others.  And 
we  might  also  want  him  to  choose  the  organization  for  his  work  career. 

These  operational  characteristics  constitute  the  effects  which  we 
wish  to  bring  about  in  our  workers.  In  this  regard,  1  strongly  re¬ 
commend  Bob  Mager's  recent  book.  Goal  Analysis,  *  as  a  guide  for 
learning  how  to  analyze  such  fuzzy  terms  as  "motivated  worker"  into 
a  set  of  operational  characteristics. 

Once  we  have  listed  the  effects  or  purposes  we  wish  to  achieve  in 
operational  terms,  then  we  need  to  select  one  or  more  suitable 
theories  of  operation  from  which  we  can  determine  what  has  to  be 
done  in  order  to  achieve  these  effects.  A  behavior  modification 
approach  accounts  for  behavior  in  terms  of  conditions  in  the  environ¬ 
ment  in  which  the  behavior  occurs.  The  most  important  environmental 
conditions  consist  of  the  manner  in  which  other  people  respond  to  the 
worker's  behavior.  In  this  instance,  these  other  people  consist  of 
his  supervisor  and  his  fellow  workers.  Whether  or  not  a  worker 
behaves  in  a  "motivated"  fashion  depends  largely  on  how  other  people 
in  his  work  environment  treat  him.  And  how  others  treat  him  depends 
to  some  extent  upon  how  he  treats  them. 

Our  problem,  as  systems  engineers,  is  to  design  a  behavior  system 
in  which  people  interact  in  such  a  way  as  to  enhance  each  other's 
work  behavior.  We  need  to  determine  the  critical  interpersonal 
behaviors  required  from  the  supervisor,  from  other  workers,  and 
from  the  worker  himself,  in  order  to  optimize  the  worker's  job 
performance.  The  inter-personal  behaviors  of  both  the  supervisor 
and  of  the  other  workers  should  accomplish  the  same  general  functions; 

*See  suggested  readings  (Appendix  C) 
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that  is  I  they  need  to  reinforce  the  worker  for  appropriate  behavior » 
refi.  in  from  reinforcing  him  for  inappropriate  behavior,  and  minimize 
aversive  stimuli  in  his  environment.  The  specific  ways  in  which 
they  go  about  performing  these  functions  will  differ. 

If  you  will  look  at  your  handouts,  you  will  find  a  statement  of  the 
supervisor's  functions  in  behavioral  terms  and  an  analysis  of  the 
purposes  which  make  up  this  function.  Again,  both  the  statement  and 
the  purposes  are  based  on  behavior  modification  principles  as  the 
underlying  theory  of  operation.  The  statement  names  the  acts  the 
supervisor  is  to  perform--he  designs  and  implements  management 
practices.  And  it  names  the  effects  such  acts  are  to  have  on  the 
behavior  of  the  people  he  manages --^strengthen  their  productive 
behaviors  and  weaken  their  counter-productive  behaviors.  In  order 
to  achieve  these  effects,  the  supervisor  must  achieve  three  major 
purposes  or  tasks: 

1.  He  must  appropriately  reinforce  productive  behaviors. 

Z.  He  must  minimize  the  inadvertant  reinforcemoit  of  counter¬ 
productive  behaviors. 

3.  He  must  minimize  aversive  conditions  in  the  environment. 

Each  of  these  purposes  can  be  analyzed  into  subordinate  purposes. 

For  instance,  in  order  to  reinforce  productive  behavior,  the  supervisor 
must: 

1 .  identify  and  define  it, 

2.  monitor  it, 

3.  identify  reinforcers, 

4.  administer  the  reinforcers,  and 

5.  modify  the  reinforcement  program,  if  necessary,  to  maintain 
or  increase  the  frequency  of  productive  behaviors. 

The  other  two  major  purposes  have  been  analyzed  in  a  similar  fashion. 
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The  overall  statement  and  its  analysis  into  different  levels  of 
purposes  constitutes  a  behavioral  model  for  prescribing  certain 
supervisor  or  leader  activities  at  all  levels  of  virtually  any 
organization.  The  situations  in  which  these  purposes  are  sought, 
however,  will  be  different  in  different  organizations  and  at  different 
levels  in  those  organizations.  Ir.  order  to  complete  the  analysis, 
we  need  to  identify  the  situations  in  which  these  purposes  are  to  be 
attained.  For  the  kinds  of  people  who  inhabit  the  particular  levels 
of  the  particular  organizations  in  which  we  are  interested,  we  need 
answers  to  the  following  kinds  of  questions: 

1.  What  kinds  of  productive  behaviors  are  required?  What 
kinds  of  things  must  subordinates  do  in  order  to  attain  the  organiza¬ 
tion's  goals  ? 

Z.  Which  of  these  productive  behaviors  typically  occur  with 
insufficient  frequency? 

3.  What  kinds  of  events  are  most  effective  as  reinforcers? 
Quite  often  the  most  effective  reinforcers  are  inter-personal 
expressions  of  approval.  But  oftentimes  supervisors  attend  to 
their  subordinates  only  to  chastise  them. 

4.  What  techniques  can  be  used  to  administer  the  reinforcers 
effectively?  Often  the  supervisor  will  have  to  be  trained  to  say  more 
approving  and  fewer  disapproving  things  to  his  subordinates. 

5.  What  kinds  of  counter-productive  behaviors  typically  occur 
Subordinates  may  fail  to  report  work  difficulties  such  as  equipment 
malfunctions,  or  subordinates  may  actually  sabotage  each  other's 
efforts. 

6.  What  kinds  of  events  typically  maintain  these  counter¬ 
productive  behaviors?  Oftentimes,  common  management  practices 
actually  reinforce  counter-productive  behavior.  Supervisors  often 
chastise  subordinates  for  reporting  work  difficulties.  And  often 

they  administer  incentives  such  as  pay  and  promotion  on  a  competitive 
basis,  and  then  chastise  their  subordinates  for  not  behaving 
cooperatively. 


7.  What  kinds  of  techniques  can  be  used  to  minimize  the  occur¬ 
red' ce  of  the  various  counter-productive  or  disruptive  behaviors?  For 
instance,  we  may  have  to  change  traditional  incentive  systems. 

8.  What  kinds  of  techniques  can  be  used  to  monitor  the  occur¬ 
rence  of  productive  and  counter-productive  behaviors?  If  you  can*t 
observe  it  and  record  it,  you  can't  manage  it. 

9.  What  are  the  typical  aversive  conditions  found  at  this  level 

of  the  organization?  We  want  to  minimize  aversive  conditions  because 
they  elicit  avoidance,  escape,  and  agressive  behaviors. 

10.  What  kinds  of  techniques  can  be  used  to  minimize  the  in- 
advertant  reinforcement  of  counter-productive  behaviors  and  the 
occurrence  of  aversive  events  ?  For  instance,  if  we  judged  "mid¬ 
night  requisitioning"  of  supplies  to  be  counter-productive,  we  would 
probably  have  to  change  the  consequences  of  military  inspections  from 
fixing  blame  to  providing  help  without  blame.  As  long  as  the  conse¬ 
quences  of  inspection  can  be  aversive  to  people's  careers,  there  will 
be  cheating  to  avoid  those  consequences. 

We  will  get  different  answers  to  these  questions  if  our  interest  is  in 
a  battalion  staff  rather  than  in  an  infantry  squad.  In  either  case,  the 
answers  can  only  be  obtained  through  the  conduct  of  field  studies 
using  interviews,  questionnaires,  direct  observation,  and  trial  appli¬ 
cations.  Note,  however,  that  these  field  studies  do  not  validate  our 
model.  They  elaborate  it.  The  model  itself  is  valid  to  the  extent 
that  our  theory  of  people  operation  and  our  information  gathering 
processes  are  valid.  In  this  case,  our  theory  of  people  operation 
consists  of  the  principles  of  behavior  modification.  Note  also  that 
the  entries  in  the  model  are  not  independent.  Although  the  entries  in 
the  lowest  level  of  the  model  look  somewhat  like  a  conventional  task 
inventory,  you  can't  pick  and  choose  from  among  them.  They  must 
be  either  accepted  or  rejected  as  a  total  package.  And,  finally,  note 
that  the  model  consists  of  several  levels,  rather  than  just  one  as 
does  the  typical  task  inventory.  This  allows  us  to  avoid  the  racking 
problem  of  selecting  a  single  level  at  which  to  write  all  our  tasks. 
Instead,  different  branches  of  our  hierarchy  can  be  analyzed  to 
different  levels,  depending  upon  our  differing  states  of  knowledge. 

To  complete  the  overall  model,  we  would  need  to  conduct  similar 
analysis  for  the  roles  of  the  other  workers  and  for  the  worker  himself. 


Similarities  in  the  Analysis  of  ^  and  **Soft**  Skills 

It  makes  no  difference  whether  we  are  dealing  with  "hard*'  or  "soft" 
skills,  the  general  systems  engineering  process  should  be  the  same. 

1.  We  identify  the  purposes  or  goals  of  the  system.  In  doing 
so,  we  begin  with  very  broad  purposes  and  analyze  these  broad 
purposes  into  successively  more  detailed  purposes  until  we  reach 

a  level  at  which  they  can  be  allocated  to  individual  workers.  Equip¬ 
ment  designers  do  some  of  this  kind  of  analysis,  but  the  systems  engi'- 
neer  will  generally  need  to  do  an  additional  analysis.  If  the  system 
doesn't  contain  any  significant  equipment,  then  the  systems  engineer 
will  have  to  conduct  the  purpose  analysis  in  to  to. 

2,  We  identify  a  suitable  theory  of  operation  for  the  equipment 
or  people  with  which  our  workers  will  interact.  The  theory  must  be 
appropriate  to  the  function  the  worker  is  to  perform  on  the  equipment 
or  people.  Sometimes  the  worker  will  perform  the  function  on  himself 
rather  than  on  someone  else.  For  instance,  target  recognition  and 
individual  decision  making  are  self  operations.  The  application  of 

the  theory  of  operation  to  the  general  analysis  of  a  gross  purpose  or 
job  function  results  in  the  development  of  a  behavioral  model  of  the 
job  function. 


3.  We  identify  the  characteristics  of  the  situation  in  which  the 
worker  will  operate  and  which  shape  the  responses  he  must  make. 
Oftentimes,  our  theory  of  operation  will  guide  us  in  the  selection  of 
appropriate  situation  characteristics.  If  we're  dealing  with  a  maintenance 
job,  we'll  want  to  identify  routine  maintenance  schedules  and  mal¬ 
function  and  symptom  information.  If  we're  dealing  with  supervisory 
Jobs,  we'll  want  to  identify  actual  reinforcing  and  aversive  events. 

4.  We  apply  our  theory  or  theories  of  operation  to  determine 
what  actions  our  worker  ought  to  take  in  order  to  attain  the  appropriate 
purposes  in  specified  situations.  If  we're  dealing  with  a  maintenance 
job,  we  may  conduct  a  trouble-shooting  analysis.  If  we're  dealing  with 
a  supervisory  job,  we  will  conduct  a  behavioral  analysis. 

The  systems  engineering  process  is  not  a  fixed  procedure  to  be 
administered  in  a  stepwise  fashion.  We  frequently  learn  something 
in  a  later  stage  that  will  cause  us  to  revise  an  analysis  done  in  an 
earlier  stage.  And,  frequently,  we  will  find  glaring  and  unsuspected 
gaps  in  the  system  or  job  that  we're  analyzing. 
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When  finished,  we  will  have  established  the  relationship  between  the 
job  *  nd  the  purposes  of  the  organization;  we  will  have  identified  a 
theory  of  operation  for  the  equipment  our  worker  will  use,  whether 
that  equipment  be  mechanical  or  human;  we  will  have  identified  the 
specific  purposes  a  worker  must  attain,  the  probable  situations  in 
which  the  purposes  will  need  to  be  attained,  and  the  actions  required 
to  attain  these  purposes  in  these  situations.  We  will  then  be  in  a 
position  to  design  job  sample  practice  and  testing  procedures.  If 
it's  a  maintenance  job,  we  may  have  students  troubleshoot  a  mal¬ 
function  inserted  in  the  equipment.  If  it's  a  supervisory  job,  we 
may  have  them  attempt  to  solve  a  work  motivation  problem  in  a  role 
playing  situation.  Instead  of  inundating  students  with  information 
about  troubleshooting  or  about  leadership,  we'll  have  them  practice 
specific  troubleshooting  or  leadership  skills  in  simulated  job  situations. 
In  the  spirit  of  the  term,  but  not  its  literal  meaning,  it's  still  "hands- 
on"  instruction  regardless  of  whether  it's  a  "hard"  or  a  "soft"  skill. 


SYSTEMS  ENGINEERING  OF  SOFT  SKILL  COURSES 


Chaplain  (LTC)  Harold  C.  Lamm 
Chief,  Curriculum  Office 
US  Army  Chaplains  School 
Fort  Hamilton,  NY  11Z52 
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My  task  this  morning  is  to  present  a  summary  of  how  systems 
engL  eering  soft  skills  was  applied  to  the  Advance  Chaplains  Course 
at  the  Army  Chaplain  School.  When  Dr.  Cruse  first  gave  me  a  call 
on  this  topic,  I  was  reminded  of  the  sermon  title  of  a  new  seminary 
graduate  shortly  after  ordination  when  he  announced  to  his  congrega¬ 
tion  that  his  sermon  title  for  the  morning  was:  **A  Brief  Liook  at  The 
Past,  Present,  and  Future  of  God,  Man,  and  the  Universe.'* 

I  will  try  not  to  be  intimidated  by  following  Dr.  Mager  and  the  HUMRRO 
presentation.  Perhaps  you'll  be  disappointed  that  this  is  not  going  to 
be  much  of  a  head  trip  that  I'm  taking  you  on  but  I  will  try  to  show  you 
what  we  did  and  how  we  did  it  at  the  Chaplain  School.  Deductions, 
conversation,  and  perhaps  criticism  as  to  the  way  we  did  it  can  take 
place  this  afternoon. 

Two  over-riding  considerations  that  we  had  in  our  approach  to  the 
systems  engineering  of  a  soft  skill  course  were:  What  do  we  want 
the  graduate  student  to  be  able  to  perform,  and  concurrently,  what 
type  of  person  do  we  want  the  graduate  to  be? 

We  are  all  familiar  with  the  seven  steps  of  the  systems  engineering 
process  as  they  appear  in  the  CONARC  regulation.  Dr.  Cruse  indicated 
that  the  concern  here  this  morning  is  with  the  first  Z-1/2  steps  of  that 
process,  taking  us  through  an  approach  to  the  writing  of  performance 
objectives  for  soft  skill  training. 

SLIDE  I 

SYSTEMS  ENGINEERING 

1.  Perform  Job  Analysis 

2.  Select  Tasks  for  Training 

3.  Prepare  Training  Analysis 

4.  Develop  Training  Materials 

5.  Develop  Evaluation  Instruments 

6.  Conduct  Training 

7.  Exercise  Quality  Control 

The  initial  work  on  the  C22  course  began  in  late  19/0  -  late  calendar 
year  1970,  and  the  systems  engineered  course  implemented  in 
September  1971.  For  illustrative  purposes,  1  will  occasionally  be 
following  through  an  example  task  from  the  genesis  of  its  identification 


to  its  current  utilization  in  the  C22  program.  The  general  CONARC 
procedures  were  followed  in  the  Job  Analysis  step  of  the  systems 
engineering  process. 

SLIDE  2 

Perform  Job 

Analysis 

As  we  sought  to  identify  on-the-job  tasks  that  Advance  Course  Chaplain 
graduates  were  expected  to  perform*  we  investigated  both  official  and 
unofficial  sources  of  information  in  order  to  identify  and  isolate  chaplain 

tasks. 

SLIDE  3 

SOURCES  OF  JOB  INFORMATION 

1.  Army  Regulations 

2.  Field  Manuals 

3.  Technical  Manuals 

4.  Directives  from  the  Office  Chief  of  Chaplains 

5.  Experienced  Job  Holders 

6.  MODB 

Among  these  sources  of  information  were  included  AR  165-20  entitled 
"Duties  of  Chaplains  and  Commander's  Responsibilities";  FM  16-5 
entitled  "The  Chaplain":  various  technical  manuals;  various  directives 
from  the  Office  of  the  Chief  of  Chaplains,  particularly  the  publication 
entitled  The  Five  Year  Program.  Another  prime  source  of  information 
was  experienced  job  holders  serving  on  the  staff  and  faculty  of  the 
Chaplain  School.  The  MODB  for  MOS  71M20  was  also  used.  I'll  have 
more  to  say  about  our  use  of  the  MODB  a  bit  later. 

These  sources  contributed  to  the  development  of  our  task  inventory. 

The  development  of  the  task  inventory  was  the  initial  block  to  be  over¬ 
come.  As  was  pointed  out  in  the  earlier  presentations,  the  identifica¬ 
tion  of  tasks,  or  the  listing  and  description  of  tasks  to  be  performed  in 
the  soft  skill  area,  is  rather  diverse  and  imprecise.  So  the  development 
of  a  task  inventory  was  the  initial  significant  step  in  the  process. 
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For  example  purpose*,  one  chaplain  task  required  by  law  is  found 
in  TM  16-5. 

SLIDE  4 


FM  16-5 

SOURCE  OF  JOB  INFORMATION 

Chapter  4,  paragraph  2a.  "The  Chaplain  is  required  by 
law  to  conduct  appropriate  public  religious  s-ervices 
for  the  command  to  which  he  is  assigned. " 

The  chaplain  is  required  by  law  to  conduct  appropriate  public  religious 
services  for  the  command  to  which  he  is  assigned.  Consequently,  this 
requirement  was  included  in  the  task  inventory.  Due  to  the  unique  job 
relationship  existing  between  the  chaplain  and  his  assistant,  the  71M 
enlisted  assistant,  the  MODB  was  screened  in  ordt  r  to  determine  task 
areas  from  which  chaplain  tasks  might  be  deduced.  Now  I  recognize 
that  the  use  of  an  MODB  for  a  task  inventory  development  for  an 
officers  career  course  might  be  somewhat  suspect.  However,  since 
the  tasks  of  the  chaplain  enlisted  assistant,  cast  in  the  hard  skill 
area,  generally  have  to  do  with  manual  tasks  in  relationship  to  a 
chaplain  task,  we  felt  the  use  of  the  MODB  was  valid  for  the  interpretive 
purposes  for  which  we  intended  it.  Therefore,  through  this  process, 
we  isolated  and  deduced  soft  skill  chaplain  tasks  from  the  71M  MODB. 

By  intensive  screening  and  analysis  of  official  and  unofficial  sources 
of  information,  a  task  inventory  of  3  major  functional  areas,  21  tasks 
groups,  86  major  duties  and  476  specific  chaplain  tasks  were  identified. 

SLIDE  5 


CATEGORIES  OF  TASK  INVENTORY 

Functional  Areas  3 

Task  Groups  21 

Duties  86 


Tasks 


476 


We  defined  functional  areas  as  groups  of  related  chaplain  functions 
supported  by  common  references  such  as  military  publications.  A 
task  group  consists  of  a  related  cluster  of  duties  that  have  similar  task 
and  task  elements  and  may  be  identified  as  an  area  of  work  within 
a  particular  duty  assignment.  A  duty  is  a  prescribed  activity  in  which 
a  chaplain  may  be  required  to  participate.  Finally,  we  thought  of  a 
task  as  the  specific  action  taken  by  an  individual  chaplain  in  performing 
a  duty. 

From  this  task  inventory,  a  survey  instrument  was  developed  for  use 
in  the  field  validation  of  this  task  list.  Surveys  were  mailed  to  over 
100  unit  chaplains  covering  the  entire  range  of  rank  and  duty  assign¬ 
ments.  In  addition  to  the  mailed  surveys,  we  desired  to  conduct  on¬ 
site  at  4  different  installations.  In  most  cases,  the  Commanders  of 
those  chaplains  were  also  interviewed  reference  the  tasks  of  their 
assigned  chaplains.  Care  was  exercised  in  order  to  Include  general 
chaplain  assignments  as  well  as  specialized  assignments,  such  as 
hospital  chaplains,  confinement  facility  chaplains,  and  chaplains  serving 
on  drug  and  alcohol  abuse  teams. 

Based  on  these  surveys  and  interview  findings,  74  tasks  were  selected 
for  school  training  from  the  total  of  476  tasks  identified. 
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TASK  SELECTION 

Total  Tasks  476 

Tasks  Selected  for 
School  Training  74 

If  we  continue  using  our  example  in  the  area  of  conducting  religious 
services,  9  tasks  were  approved  and  selected  for  school  training  out 
of  a  total  of  33  tasks  for  the  conduct  of  religious  service  areas. 

SLIDE  7 


TASKS  RELATING  TO 
CONDUCTING  RELIGIOUS  SERVICES 

Total  33 

Selected  for  School  Training  9 
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The  selection  of  tasks  for  training  was  performed  at  the  Chaplain 
School  by  an  Ad  Hoc  Systems  Engineering  Committee.  This  Committee 
was  composed  of  representatives  of  the  Office  of  Director  of 
Instruction  (Curriculvim  and  Evaluation  and  Methods),  the  Education 
Advisor,  and  members  of  the  staff  and  faculty.  Selections  were  made 
on  the  basis  of  the  standard  regulatory  criteria.  But  a  key  tool  used 
in  the  selection  process  was  the  completed  task  validation  form,  an 
example  of  which  is  shown  here. 

SLilDE  8  (Appendix  D) 

In  order  to  transform  the  tasks  selected  for  school  training  into  train* 
ing  objectives,  the  resident  factdty  was  organized  into  committees 
corresponding  to  the  three  major  functional  groupings  of  tasks  selected. 
I  might  add  at  this  point  of  the  systems  engineering  process,  as  we 
identified  the  three  major  functional  areas,  the  Chaplain  School 
Resident  Department  was  restructured  to  align  itself  with  these  three 
functional  areas.  These  three  functional  groupings  were  the  pastoral 
activities,  management,  and  the  military  operations  and  organization 
areas. 

Each  functional  group  of  selected  task  was  presented  to  the  correspond¬ 
ing  committee  whose  responsibility  it  was  to  develop  the  training 
objective.  Through  this  committee  effort,  we  directly  involved  the 
faculty  in  the  systems  engineering  process,  thereby  providing  an 
opportunity  for  intellectual  and  ego  investment  on  the  part  of  the  Faculty 
into  the  systems  engineering  effort. 

Again,  by  way  of  illustration,  using  our  "conduct  religious  services" 
task  model,  while  translating  the  selected  tasks  for  school  training 
into  training  objectives,  we  see  progress  from  a  task  identified  in  a 
field  manual,  "the  chaplain  is  required  by  law  to  conduct  appropriate 
public  religious  services  for  the  command  to  which  he  is  assigned" 
to  seven  performance  objectives  selected  and  approved  for  school 
training. 
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TITLES  OF  TRAINING  OBJECTIVES  RELATING  TO 
CONDUCTING  REUGIOUS  SERVICES 


2Z020  Research  currert  literature  for  sermons, 
homilies,  anc  meditations 

22021  Employ /Support  others  in  employing  new 

forms  of  worship 

22022  Provides  for  the  administration  of  sacra¬ 

ments  and  rites 

22023  Provide  lay  participation  in  worship 

22024  Provide  opportunity  to  meet  Jewish  Holy 

Day  requirements 

22025  Provide  interfaith  worship  opportunities 

22026  Provide  worship  opportunities  for  Non 

4udaeo-Christian  Groups 

These  seven  objectives  were  developed  out  of  this  one  task  identified 
in  the  field  manual. 

We  have  learned  several  things  from  this  exercise  in  systems 
engineering  of  soft  skill  training  at  the  Chaplain  School. 

SLIDE  10 

LESSONS  LEARNED 

1.  Dealing  with  attitudes 

2.  Faculty  resistance 

3.  Level  of  specificity 

4.  Student  Centered  Instruction 

Basically,  for  summary  purposes,  four  major  lessons  learned  areas 
are  those  relating  to  the  problem  of  dealing  with  attitudes,  staff  and 
faculty  resistance,  the  matter  of  determining  the  proper  level  of 
specificity  of  training  objectives,  and  the  area  of  student-centered 
instruction. 
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It  is  to  plow  old  ground  to  say  there  is  difficulty  involved  in  teaching 
an  altitude.  When  an  attitude  was  identified  in  the  systems  engineering 
process,  the  task  of  teaching  that  attitude  or  rather  changing  behavior 
to  conform  to  that  attitude,  presented  the  usual  problems.  We  found 
that  as  the  small  group  was  the  optimal  means  for  task  analysis,  task 
selection,  and  for  the  writing  of  objectives;  so  also,  we  found  small 
groups  to  be  the  best  means  available  to  deal  with  the  matter  of 
attitudes. 

In  the  area  of  staff  and  faculty  resistance,  academic  inertia  and  resis¬ 
tance  to  change  was  as  prevalent  at  the  Chaplain  School  as  at  nearly 
any  other  military  or  civilian  educational  institution.  We  might  say 
that  resistance  to  change  is  randomly  distributed  in  the  educational 
community.  As  the  task  selection  committee  met,  their  orientation 
to  the  subjective  element  within  the  systems  engineering  process  (by 
subjective  elements  within  the  systems  engineering  process  I  mean 
those  experiences  that  the  job  holders  themselves  were  able  to  bring 
in  to  the  systems  engineering  process)  relieved  much  of  the  initial 
resistance.  I  don't  want  to  get  too  deeply  involved  into  the  etiology 
of  the  resistance  but  part  of  it  was  based  on  the  impression,  or  the 
lack  of  clear  definition  or  impression,  of  what  the  systems  engineer¬ 
ing  process  was  as  it  related  to  the  soft  skill  area.  It  was  somehow 
perceived  as  a  totally  impersonal,  objective  process  which  either  de¬ 
valued  or  did  not  take  into  account  the  accumixlated  professional  ex¬ 
perience  of  the  members  of  the  staff  and  faculty  themselves.  When 
they  saw  the  possibility  of  their  own  input  and  the  increased  possibility 
of  student  feedback,  much  of  the  initial  faculty  resistance  was  overcome. 

During  the  first  writing  of  training  objectives  we  found  that  we  had 
written  the  objectives  at  too  exact  or  too  high  a  level  of  specificity. 

For  example,  seven  objectives  were  initially  developed  in  the  worship 
area  from  the  selected  task  of  "conduct  of  religious  services, "  and 
each  objective  was  taught  as  a  discrete  block  of  instruction.  Both 
students  and  faculty  recognized  that  these  seven  objectives  should  be 
incorporated  into  one  objective,  which  has  been  utilized  to  construct 
the  current  block  of  instruction.  I  would  like  to,  at  this  time,  dis¬ 
tribute  this  particular  objective  as  written.  An  objective  on  religious 
coverage  will  also  be  distributed.  ^ 

^Handouts  are  available  upon  request  from  the 
US  Army  Chaplain  School 
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These  objectives,  as  you  later  look  at  them,  will  lead  you  to  the  fourth 
area  of  our  "lessons  learned, "  and  that  is  the  area  of  student  centered 
instruction.  Our  career  class  has  been  organized  into  eight  learning 
groups  of  twelve  students  and  one  faculty  advisor.  It  is  the  current 
practice  at  the  Chaplain  School  to  have  the  primary  instructor  prepare 
the  training  objective  within  the  format  of  the  performance  actions, 
conditions  and  standards.  This  training  objective  is  then  presented 
to  a  board  consisting  of  the  eight  student  faculty  advisors.  This  board 
of  facility  advisors  serves  as  a  type  "murder  board"  and  is  permitted 
to  modify  the  objective.  This  objective  then  is  presented  to  the  students 
At  the  time  of  presentation,  the  students  are  also  free  to  modify  the 
means  of  achieving  the  objective.  They  cannot  modify  the  terminal 
required  performance  actions  and  standards,  but  latitude  is  given  to 
the  students  to  modify  the  means  of  achieving  that  performance  action. 
Through  these  modification  procedures,  we  are  thus  attempting  to 
have  the  systems  engineering  process  opened  to  immediate  feedback 
and  we  are  also  seeking  to  have  an  immediate  feedback  procedure  as 
part  of  the  learning  environment  within  the  classroom  setting  itself. 

We  are  currently  conducting  our  second  systems  engineered  career 
course.  The  class  last  year  assisted  in  the  re-engineering  process 
of  the  first  class  and  we  expect  the  class  this  year  will  serve  a  similar 
function.  To  say  that  the  class  last  year  assisted  in  modification  and 
that  we  intend  to  do  the  same  this  year,  suggests  that  we  don't  feel 
satisfied  that  we  have  found  all  the  answers  to  all  the  problems.  This 
is  true.  We  recognize  it  and  we  feel  it  is  "O-K"  since  we  regard 
systems  engineering  of  instruction,  particularly  soft  skill  instruction, 
as  an  on-going  continuing,  dynamic  process. 

This,  in  short,  has  been  what  we  did  at  the  Chaplain  School,  and 
hopefully  gave  you  a  bit  of  a  look  at  how  we  did  it.  Thank  you. 
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A  FUNCTIONAL  APPROACH  IN  DESIGN  OF 
SOFT  SKILL  TRAINING 


Captain  Yvonne  N.  Burger 
Curriculum  and  Evaluation  Branch 
US  Army  Women's  Army  Corp  School 
Fort  McClellan,  Alabama  36Z01 


One  of  the  most  trying  problems  in  attempting  to  design  Officer  and 
NCO  professional  development  courses  is  encountered  in  identifying 
just  exactly  what  we  are  trying  to  teach  students  to  DO,  Every 
approach  tends  to  come  to  the  same  point:  instead  of  teaching  them 
to  DO,  we  are  trying  to  teach  ther  i  to  BE  something  --  military  leaders 
and  managers  and  the  many  things  they  DO  as  leaders  and  managers 
depends  upon  the  circumstances  oi  the  particular  situation.  It  would 
seem  more  realistic,  then,  to  focus  on  the  common  functions  of  manage • 
ment  and  leadership,  rather  than  try  to  predict  precisely  which  tasks 
a  manager  or  leader  would  be  most  likely  to  perform  and  under  what 
particular  set  of  circumstances. 

I  would  like  to  present  a  cybernetic  model  of  management  as  one 
approach  to  identifying  the  functions  of  management/leadership.  A 
manager  is  seen  as  the  controller  of  a  system.  This  system  can  be 
at  any  level  of  complexity  --  from  one  person  with  one  task  to  many 
people  with  many  complex  functions.  A  system  equates  roughly  to  a 
job  for  purposes  of  discussion.  An  example  of  a  cybernetic  system  is 
shown  in  Appendix  E.  In  this  model,  there  are  four  main  parts:  (1)  the 
controller  (manager);  (2)  the  methods  and  means;  (3)  the  goals  (outputs); 
(4)  the  feedback  system,  which  tells  the  controller  the  effects  of  the 
action  taken.  In  using  the  model  it  is  highly  important  to  consider  the 
relationships  of  one  system  to  another  (interfaces)  and  the  constraints 
or  limitations  which  apply  to  any  given  system. 

Using  the  suggested  approach  in  systems  engineering  soft-skill  training 
courses  for  managers,  it  would  be  necessary  to  determine  the  functions 
of  the  controller  of  our  model  system  and  identify  the  skills  and  knowl^ 
edges  needed  by  the  controller  to  perform  these  functions.  The  functions 
of  the  system  controller  in  this  model  would  be  to:  (1)  learn  the  goals 
>f  that  particular  system;  (2)  establish  the  communications  required  to 
activate  and  control  the  system  and  to  receive  the  necessary  feedback 
in  a  useful  time  frame;  (3)  direct  the  methods  and  means  necessary  to 
accomplish  the  goals;  and  (4)  establish  procedures  whereby  the  system 
can  operate  efficiently. 

Skills  needed  by  the  controller  to  perform  Ihe  functions  of  our  model 
system  are  ability  to  make  decisions,  plan,  organize,  define,  and 
evaluate  situations;  set  priorities;  properly  utilize  resources;  exercise 
judgment  in  initiating  and  stopping  actions;  evaluate  actions  taken  in 


terms  of  the  systems'  goals;  set  standards;  and  maintain  the  discipline 
of  th*.  system.  Knowledge  needed  to  perform  the  functions  of  the  system 
include  knowledge  of  interface  relationships »  procedures,  tools,  and 
constraints  of  the  system. 

In  general,  there  seem  to  be  three  major  areas  of  management  expertise 
involved  in  efficiently  controlling  a  system:  (1)  management  of  people 
(leadership);  (2)  knowledge  of  the  processes,  procedures,  and  tools 
involved;  (3)  technical  knowledge.  Major  areas  1  and  2  above  will  be 
common  to  every  job  an  Officer  or  NCO  holds  throughout  his  career; 
therefore,  it  would  seem  that  these  are  the  two  areas  which  should  be 
taught  in  professional  development  courses. 

An  advantage  of  such  an  approach  would  be  to  change  the  nature  of  the 
Officer /NCO  self-image.  An  experiment  which  I  conducted  with  some 
newly  graduated  Officer  Basic  Course  Students  showed  that  they  did 
not  have  any  idea  what  their  respcnsibilities  would  be  when  they  reached 
their  new  assignments.  Typical  answers  to  the  question,  "What  are 
you  going  to  do  first  when  you  get  to  your  new  duty  station?"  were  "I 
don't  know"  and  "It  will  depend  on  what  job  I  end  up  with. "  If  they  had 
thought  of  themselves  as  managers  of  a  system,  representative  answers 
might  well  have  been  "I  will  really  be  busy.  I  have  to  learn  what  my 
job  is;  what  resources  I  have  available;  all  about  the  people  I'll  be 
working  with;  what  the  parameters  of  my  job  are;  what  my  interfaces 
are;  what  kind  of  communications  system  I'll  need  to  keep  things 
moving;  and  what  procedures  and  tools  are  used  in  varying  situations". 

One  of  the  goals  of  such  functionally-oriented  training  would  be  to 
permit  students  to  demonstrate  their  skills  of  management,  rather 
than  have  them  read  and  state  the  principles  of  management  and  hope 
that  they  could  apply  the  principles  properly  in  future  assignments. 

The  problem  of  qualifying  officer  and  NCOs  in  specific  skill  areas 
should  be  accomplished  by  MOS  courses,  correspondence  courses, 
and  OJT  as  it  presently  is,  thus  allowing  the  professional  development 
courses  to  concentrate  on  the  skills  and  knowledges  of  management 
and  leadership.  These  courses  should  provide  practice  in  skills  such 
as  decision-making,  organizing,  initiating  and  evaluating  actions, 
and  other  managerial  functions;  they  should  also  teach  him  how  to 
determine  what  he  needs  to  know  to  effectively  undertake  the  respon¬ 
sibilities  of  a  new  job. 
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In  general^  the  student  would  see  himself  as  a  manager  --a  decision¬ 
maker  --  and  as  the  director  or  "controller"  of  a  system.  He  would 
know  that  he  must  understand  the  system  he  is  to  direct.  He  would 
know  that  he  must  learn  its  boundaries,  its  limitations,  and  its 
interface  with  other  systems.  He  would  know  that  he  must  learn  the 
goals  of  the  system,  the  methods  and  means  available  to  him  for 
accomplishing  those  goals,  and  the  communications  channels  and 
processes  required  to  keep  the  system  functioning  smoothly. 


SYSTEMS  ENGINEERING  OF  THE  ORDNANCE 
OFFICER  ADVANCED  COURSE 


Mr.  William  E.  Davis 
United  States  Army  Ordnance  School 
Aberdeen  Proving  Ground 
Aberdeen,  Maryland  21005 


In  November  1970f  the  Ordnance  School  began  a  resystems  engineeriilg 
of  the  Ordnance  Officer  Advance  Course*  This  course  was  originally 
systems  engineered  during  1966-67  when  we  converted  from  a  six 
to  a  nine  month  course  to  comply  with  the  Haines  Board  recommenda¬ 
tions*  The  revision  was  accomplished  in  minimal  time,  and  at  a  time 
when  all  personnel  were  not  familiar  or  comfortable  with  the  new 
Systems  Approach  to  course  design.  As  a  result,  many  staff  and 
faculty  seriously  questioned  the  validity  of  the  content  of  the  new  course* 
Many  minor  revisions  were  made  in  individual  subject  areas,  based 
on  feedback,  during  the  subsequent  3  year  period,  yet  this  concern 
persisted* 

Presently  we  are  putting  the  finishing  touches  on  the  training  analysis 
step  of  the  new  course*  Progress  has  been  sporadic  due  to  personnel 
shortages,  workload,  and  other  higher  priority  tasks.  We  have  now, 
however,  given  this  effort  increased  emphasis  and  have  established 
relatively  firm  target  dates  for  future  steps  and  hope  to  have  the  draft 
POI  not  later  than  the  end  of  this  fiscal  year*  While  we  believe  we 
have  done  a  good,  job  so  far,  we  must  now  speed  up  the  action  because 
time  is  taking  its  toll.  This  is  particularly  critical  in  that  turn  over 
of  personnel  is  causing  difficulty  in  maintaining  continuity  of  effort. 

To  accomplish  systems  engineering  of  the  advance  course,  the  Command 
and  Staff  Training  Department  designated  a  full  time  project  officer  - 
An  experienced  LTC  of  ordnance*  Working  with  the  department  director, 
his  assistant  and  other  key  personnel,  the  project  officer  approached 
job  analysis,  the  most  difficult  systems  step,  from  the  standpoint  of 
strategy.  Unlike  an  officer  or  enlisted  MOS  course,  we  could  not 
simply  study  the  MOS  descriptions,  the  various  duty  assignments  and 
job  performed  to  derive  a  task  inventory*  Rather,  we  were  faced  with 
the  course  purpose  shown  here: 

SLIDE  1 


PURPOSE  OF  ADVANCED  COURSE 

"To  prepare  ordnance  officers  for  command  and  staff  duties 
at  battalion  through  brigade  or  comparable  levels  in  both 
divisional  and  nondivisional  units,  with  emphasis  on  the 
exercise  of  command  at  battalion  level,  and  to  perform 
integrated  materiel  management  duties  appropriate  for  field 
grade  officer  positions,  with  emphasis  on  assignments  within 
the  CONUS  Industrial  Mobilization  Base* " 
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The  first  part  of  the  purpose  you  will  recognize  as  the  standard  CONARC 
sta^  >ment  for  an  advance  course.  The  underlined  portion  was  added  by 
our  Commandant  to  direct  our  efforts.  We  must  prepare  the  Ordnance 
Officer  for  a  wide  variety  of  jobs*  to  include  AMC-DSA  type  assign¬ 
ments,  because  the  advanced  course  is  the  last  ordnance  oriented 
course  he  will  attend  in  his  career. 

This  purpose  leads  directly  to  no  single  MOS.  Rather,  course  intelli¬ 
gence  supplied  by  OPO  showed  that  2300  assignments  of  Ordnance 
Officers  in  grades  of  major  thru  colonel  reflected  approximately  88 
different  MOS's.  The  implied  purpose  of  the  advance  course  is  to 
provide  a  broad  educational  base  for  all  future  branch  assignments. 

We  therefore  concluded  that  we  must  design  a  course  to  prepare  the 
student  for  the  most  frequent  or  typical  assignments  and  would  have 
to  ignore  many  MOS  areas. 

A  thorough  review  of  assignments  led  to  the  selection  of  eight  MOS's 
which  then  became  the  basis  for  all  subsequent  efforts.  Those  MOS's 
are  shown  here: 

SLIDE  2 

MAJOR  ASSIGNMENT  AREAS 

MOS  2162  Operations  Officer 

MOS  2167  RtcD  Coordinator 

MOS  4010  Supply  Staff  Officer 

MOS  4011  Maintenance  Staff  Officer 

MOS  4319  Procurement  Control  and  Production  Officer 

MOS  4320  Procurement  Officer 

MOS  4515  Missile  and  Munitions  Officer 

MOS  4803  Maintenance  Officer 

These  MOS's  were  determined  to  cover  the  vast  majority  of  all  ordnance 
officer  assignments  and  had  many  duties  common  to  most  of  the  80  not 
included. 

At  this  point  a  systems  engineering  committee  was  appointed  consisting 
of  the  previously  tMntioned  project  officer  as  chairman,  his  alternate, 
and  6  field  grade  o'nicers  with  MOS's  and  experience  in  the  above  areas. 
These  officers,  all  from  CIcSTD,  have  since  functioned  on  a  part  time 
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basis.  Consideration  was  given  to  making  this  a  full  time  duty  to  speed 
up  the  effort,  however,  present  and  projected  resources  were  such  that 
it  was  not  practical  to  pull  this  many  key  people  from  their  primary 
duties  as  branch  chiefs  and  instructors.  In  addition,  the  department 
director,  his  civilian  assistant,  and  the  senior  ordnance  L>TC  in  the 
department  acted  as  a  review  board,  reviewing  and  approving  accom¬ 
plishments  of  the  committee. 

The  systems  engineer  committee  then  prepared  the  task  inventory  - 
developing  a  separate  task  list  for  each  MDS  area.  Redundancy  was 
immediately  recognized  in  this  approach.  Therefore,  tasks  dealing 
with  things  such  as  unit  and  personnel  administration,  tasks  common 
to  all  MOS's  were  included  in  only  one  of  the  MOS  task  lists. 


A  major  difficulty  arose  concerning  the  level  of  specificity  of  task 
identification.  I  will  show  some  examples  later,  but  will  now  discuss 
the  problem  in  rather  general  terms.  In  some  areas  such  as  the  hard 
core  ordnance  areas  of  supply  and  maintenance,  we  found  it  rather  easy 
to  develop  a  detailed  task  inventory;  one  in  which  there  was  a  high  degree 
of  agreement  regarding  the  validity  of  the  inventory.  In  many  respects 
the  job  analysis  for  these  areas  was  very  similar  to  that  for  hard 
skill  officer  courses  such  as  the  Ordnance  Officer  Maintenance  Course, 
MOS  4815.  Some  examples  of  these  tasks  are,  approve /disapprove 
quarterly  reports  of  operation  loss  or  breakage,  maintain  hand  receipt 
files,  »nd  maintain  organizational  document  registers. 

In  the  R&O  and  procurement  areas  we  faced  a  difference  situation. 
Initially,  we  were  not  able  to  identify  the  tasks  with  any  great  degree 
of  detail.  For  example,  we  included  many  tasks  that  contained  the 
action  verbs  "supervise",  "monitor",  and  "manage".  When  we 
attempted  to  identify  what  an  officer  does  when  he  supervises  and  manages 
we  foxmd  a  high  degree  of  variability  from  position  to  position  and  from 
task  to  task.  Let  me  illustrate.  The  task  Supervise  Civilian  Personnel 
varies  along  many  dimensions  such  as  the  grade  range  of  the  civilians, 
their  category,  i.  e. ,  GS  or  wage  grade,  and  professional  standing. 

This  is  an  example  where  everyone  that  has  supervised  civilians 
will  have  many  experiences  to  contribute.  Reaching  a  consensus  with 
respect  to  the  relevant  and  significant  variables  is  not  easy.  This  is 
where  we  encountered  the  greatest  difference  between  hard  skill  and 
soft  skill  areas.  In  general,  our  approach  has  been  to  accept  initially 


a  rather  general  task  statement  and  then  proceed  to  more  detailed 
behaviors  much  the  same  as  we  do  when  identifying  skills  and  knowl¬ 
edges  for  hard  skill  tasks.  This  suggests  to  us  that  in  the  truly  soft 
skill  areas,  maybe  it  is  the  skills  and  knowledges  that  become  the 
real  training  requirements.  When  they  are  integrated  in  actual  practice 
such  tasks  or  concepts  as  "supervise”  or  "manage"  emerge.  Here, 

I  am  suggesting  an  approach  in  which  the  whole  is  greater  than  the  sum 
of  its  parts. 

We  can  teach  an  officer  to  prepare  a  staff  study  and  to  analyze  reports 
for  deficiencies  and  clues  to  needed  actions,  but  such  tasks  as  supervise 
and  manage,  leave  room  for  uncertainty.  We  start  with  such  tasks 
and  proceed  to  break  them  into  their  component  elements.  An  example 
from  the  R&D  coordinator  was  the  task  Monitor  Design  Activities. 

We  then  specifically  broke  this  down  into  things  like:  Prepare 
Memoranda  Indicating  Design  Status;  Make  Recommendations  Relative 
to  Design  Concept,  etc. 

During  the  second  step  of  system  engineering.  Selecting  Tasks  for 
Training,  we  encountered  certain  different  considerations.  In  the 
areas  of  maintenance  and  supply,  we  were  able  to  select  tasks  in  a 
manner  similar  to  that  which  had  been  used  with  hard  skill  courses. 
Since  the  tasks  were  rather  specific,  we  encountered  no  major  difficul¬ 
ties  and  our  selection  criteria  were  basically  as  stated  in  CON  Reg 
350-100-1. 

In  the  R&D  and  procurement  areas  we  found  it  necessary  to  change 
the  rationale  for  task  selection.  The  major  reason  being  that  many 
officers  will  receive  additional  training  prior  to  assignments  in  these 
areas  such  as  the  Project  Manager  Course  at  Ft.  Belvior,  VA.  In 
this  case  we  found  that  our  primary  interest  was  more  one  of  sampling 
in  a  general  way  the  more  numerous  R&D  and  procurement  tasks.  In 
this  manner,  we  hope  to  provide  a  broad  foundation  that  will  serve  as 
a  useful  base  on  which  to  build  specific  performance  capabilities.  The 
point  I  wish  to  emphasize  is  that  selecting  tasks  for  training  in  soft 
skill  areas  requires  that  more  thought  be  given  to  the  underlying 
philosophy  of  task  selection.  In  hard  skills  this  philosophy  seems  to 
remain  rather  constant  and  the  list  of  specific  criteria  can  be  followed 
rather  routinely.  With  soft  skills,  this  underlying  philosophy  must  be 
re-examined  as  the  selection  process  moves  from  area  to  area,  and 
the  overall  objective  in  each  area  must  always  be  visible. 
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Despite  efforts  previously  mentioned  to  minimize  redundancy,  our 
approach  revealed  considerable  repetition  of  tasks  among  MOS  areas. 

Of  more  than  2,  000  tasks  originally  identified,  we  selected  1,  039 
for  training.  To  these  tasks  we  added  those  CONARC-DA  Essential 
Training  Subjects  not  included  in  our  task  inventory  and  gave  considera¬ 
tion  to  CONARC-DA  Selected  Tr.-ining  Subjects  for  appropriateness 
in  our  course.  Here  again  the  inventory  was  reviewed  and  approved. 

The  next  step  in  systems  engineering,  training  analysis,  has  proved 
to  be  the  most  laborious,  if  not  the  most  difficult  step  thus  far  encoun¬ 
tered.  The  information  gathered  in  the  task  inventory  led  directly  to 
the  identification  of  supporting  skills,  knowledges,  and  job  significant 
attitudes.  To  assist  administratively,  job  standards  and  job  condition 
codes  were  established  for  each  MOS. 

Developing  training  objectives  and  evaluation  criteria  is  another  matter. 
As  many  of  you  know,  writing  a  useful,  three-part  training  objective 
with  task,  condition,  and  standard  is  not  easy. 

Several  factors  contribute  to  this  difficulty.  Some  of  the  more  significant 
ones  are: 

1.  People  require  intensive  training  and  practice  in  writing 
behaviorally  oriented  training  objectives.  It  doesn't  come  naturally. 

2.  Inadequacies  may  appear  in  the  job  analysis.  These  include 
insufficient  information  and  inappropriately  defined  task  statements. 

3.  Coordination  and  correlation  with  various  groups  is  essential 
to  insure  agreement  on  identified  common  elements.  This  is 
particularly  difficult  to  accomplish  with  a  high  turnover  of  personnel. 

Now  I  will  provide  examples  of  objectives  tla  t  sample  the  spectrum 
that  1  have  been  describing. 

One  of  the  simpler  tasks  in  taken  from  the  MOS  4011  Maintenance  Staff 
Officer  inventory  dealing  with  supply  procedures.  The  task  being 
simply: 
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TASK 

Approves /Disapproves  Reports  of  Survey 
OBJECTIVE 

In  a  classroom  environment  the  student  will  review  and 
approve  or  disapprove  reports  of  survey.  Reports  will 
be  evaluated  against  criteria  contained  in  AR's  710-2, 

735-10,  and  735-11.  Specific  deficiencies  will  be  listed 
and  students  will  approve,  approve  with  minor  modifi¬ 
cations,  or  disapprove. 

The  training  objective  developed  for  this  task  is  basically  straight 
forward  requiring  the  student  to  actually  review  reports  of  survey 
and  decide  for,  or  against  approval  based  upon  army  regulations. 

Once  we  identified  the  task  to  this  level,  the  training  objective  was 
not  too  difficult.  However,  I  wish  to  emphasize  that  this  task  is  part 
of  a  larger  requiremert  of  supervising  unit  supply  procedures.  If 
we  compare  to  hard  skill  courses,  we  might  say  this  is  a  skill  and 
knowledge  that  is  common  to  many  job  requirements.  This  broad 
spectrum  requirement  is  satisfied  by  that  portion  of  the  objective 
which  specifies  both  garrison  and  field  conditions. 

The  next  slide  illustrates  a  slightly  more  complex  task. 
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TASK 

Supervise  the  Operation  of  Automatic  Supply 
OBJECTIVE 

In  a  command  post  exercise  environment  using  doctrine 
published  in  AR  710-1  and  2,  the  student  will  analyze 
reports  of  activity  supply  operations  and  will  identify 
deficiencies  and  recommend  corrective  action. 
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This  objective  is  of  particular  interest  because  we  were  able  to 
redefine  Supervise  Operations  o  '  Automatic  Supply  as  Analyze 
Operations,  Identify  Deficiencies  and  Recommend  Corrective  Actions 
if  required.  We  recognize  that  ether  actions  are  necessary  such 
as  Developing  Automatic  Supply  Procedures,  but  such  requirements 
are  provided  for  as  part  of  other  objectives. 

The  next  task  was  taken  from  the  missile  and  munition  inventory,  but 
could  come  from  most  any  duty  MOS  dealing  with  management  of 
personnel  within  an  organization.  This  task  is  broader  than  the  first 
two. 
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TASK 

Manage  Organization  Personnel 
OBJECTIVE 

In  a  classroom  environment,  the  student  will  review  personnel 
records,  recommend  assignments,  and  evaluate  performance 
of  organizational  personnel.  Standards  of  acceptable  perfor¬ 
mance  are  contained  in  AR'e  600-200,  600-8,  600-16,  600-20, 
611-1,  614-1,  611-15,  60-21,  612-200,  and  230-2,  and 
simulated  local  SOP's. 

Using  approved  DA  publications,  we  break  down  "manage"  into  Review¬ 
ing  Records,  Recommending  Assignments,  and  Evaluating  Organiza¬ 
tional  Personnel.  Again,  not  all  implied  subtasks  are  involved,  those 
not  covered  in  this  objective  are  covered  in  others. 


The  last  task  we  will  discuss  is  from  R8cD  area: 
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TASK 

Mftke  Recoininendfttione  for  Entry  Into  Engineering 
Development  or  Contract  Definition 

OBJECTIVE 

In  a  classroom  environment  the  student  will  develop  an  outline 
plan  which  identifies  the  actions  which  must  be  completed  before 
a  development  project  can  move  into  Engineering  Development 
or  Contract  Definition  (i.  e. ,  Concept  Formulation  through 
approval  of  MN(ED)  must  be  complete).  Job  Standards  A  thru 
P  apply. 

This  broad  task  encompasses  many  skills  and  knowledges  relating  to 
the  life  cycle  management  of  army  materiel  and  status  of  RiD  projects. 
Here  the  student  must  develop  an  outline  plan  and  identify  the  actions 
which  must  be  completed^  before  recommendations  can  be  made  to 
enter  into  engineering  development  or  contract  definitions.  In  this 
way,  the  general  task  Make  Recommendation  is  reduced  to  a  more 
specific  task  or  behavior. 

While  we  do  not  consider  the  objectives  you  have  just  seen  to  be  perfect 
models  for  soft  skill  tasks,  we  do  believe  they  are  workable.  While 
not  a  part  of  this  discussion,  many  of  the  problems  encountered  in 
development  of  training  objectives  also  occured  when  writing  the 
accompanying  criterion  statement.  However,  if  the  objective  was  well 
stated  this  usually  led  to  a  good  criterion  statement. 

Systems  engineering  of  our  advanced  course  has  been  a  long  laborious 
task.  We  have  learned  much,  not  the  least  being  a  great  deal  about 
systems  engineering.  Among  the  significant  lessons  learned  are: 

1.  Serious  Consideration  should  be  given  to  forming  a  full  time 
committee.  As  stated  earlier,  we  considered  this  but  at  the  time 
believed  we  could  not  afford  it.  Some  now  believe  it  may  have  been 
less  costly  in  the  long  run  to  have  lormed  a  full  time  committee. 
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2.  Don't  underestimate  the  need  for  training.  Many  of  our 
di^icultiea  have  been  directly  traceable  to  a  lack  of  understanding  of 
the  system  engineering  process  on  the  part  of  the  people  involved. 

3.  Even  though  there  are  differences  between  hard  skill  and 
soft  skill  courses  these  differences  do  not  preclude  the  use  of  the 
system  engineering  approach  when  developing  soft  skill  courses. 

In  conclusion,  while  still  not  complete,  we  can  see  a  new  advanced 
course  over  the  horizon.  It  may  not  be  too  radically  different  from 
our  present  course,  but  after  this  effort  we  will  be  more  certain 
that  it  has  the  validity  required  to  meet  the  needs  of  our  ordnance 
officers  in  their  future  assignments. 


SYSTEMS  APPROACH  IN  DESIGNING  COURSES 
FOR  EIGHT  COMBAT  MOS's 


Lieutenant  Colonel  Mike  Lyman 
United  States  Army  Infantry  School 
Fort  Banning,  Georgia  31905 


Introduction^  I  am  LTC  Mike  Lyman  from  the  Infantry  SchooL 
Fort  Benning.  I  have  been  in  charge  of  the  systems  engineering  of 
resident  courses  during  the  past  three  years  until  July  197Z.  Currently, 
I  have  the  Training  Division  of  the  Office  of  Doctrine  Development 
Literature  and  Plans,  I  am  in  charge  of  the  Infantry  School  systems 
engineering  group  that  is  performing  the  Combat  Arms  Training  Board's 
Eight  Combat  MOS  Study, 

Purpose  of  Briefing, 

SLIDE  1 


PURPOSE  OF  BRIEFING 

Discuss  rationale  for  eight  combat  MOS  study 
Describe  procedures  and  considerations  used  in 
systems  engineering  soft  skill  tasks 

The  general  purpose  of  my  briefing  is  to  provide  you  some  insight  for 
the  Eight  Combat  MOS  Study*  the  procedures  we  are  using  and  how  we 
are  handling  soft  skill  systems  engineering.  More  specifically,  I  will 
discuss  rationale  for  the  Eight  Combat  MOS  Study  and  describe  pro¬ 
cedures  and  considerations  used  in  system  engineering  soft  skill  tasks. 

Concept  of  Eight  Combat  MOS  Study.  In  August  of  this  year  the  Combat 
Arms  Training  Board  (CATB)  decided  to  systems  engineer  two  key  MOS's 
from  each  of  the  four  combat  arms.  Infantry,  Armor,  Artillery,  and  Air 
Defense.  It  was  felt  that  there  was  a  body  of  common  tasks  performed  by 
personnel  in  each  of  these  MOS's.  The  MOS's  selected  are  shown  in 
this  slide. 

SLIDE  2  (Appendix  F) 

Each  school  involved  selected  several  duty  positions  from  each  MOS  at 
the  various  skill  levels  through  skill  level  "4"  and  commenced  preparing 
a  job  identification  as  outlined  in  the  CONARC  regulation.  For  example, 
eleven  duty  positions  in  MOS  IIB  were  selected.  Each  school  then 
separately  developed  a  task  inventory  for  their  two  MOS's  and  cross- 
referenced  them  to  a  matrix  containing  the  duty  positions.  In  September 
the  four  schools  got  together  and  compared  their  listings.  After  a  nine 
day  session,  the  common  tasks  performed  by  at  least  one  duty  positron 
in  each  of  the  eight  MOS  were  identified  and  agreed  upon.  These  tasks 
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were  called  fundamentals  and  were  categorized  by  common  subject 
areas  The  remaining  tasks  were  reorganized  into  branch  and  MOS 
specific  tasks.  Those  classified  as  branch  were  those  remaining  tasks 
that  were  performed  by  at  least  one  duty  position  in  both  MOS's  of 
the  branch.  The  remaining  tasks  were  grouped  by  category  into  MOS 
specific  tasks. 

Common  Categories.  The  categories  having  tasks  that  were  common 
to  all  eight  of  the  MOS  are  showri  in  this  slide. 

SLIDE  3  (Appendix  G) 

Although  initially  we  had  identified  43  categories »  only  the  28  shown 
were  common  to  at  least  one  duty  position  in  all  eight  MOS.  Following 
the  meeting  CATB  designated  certain  of  these  common  categories 
for  development  by  selected  combat  arms  schools.  In  many  categories 
however,  each  school  had  branch  and  MOS  specific  tasks  to  develop. 

In  addition  several  categories  that  were  not  common  to  all  eight  MOS 
were  to  be  developed.  Currently  each  school  is  developing  Job  Task 
Data  for  the  common  categories.  Job  Task  Data  as  I'm  sure  most  of 
you  know  consists  of  subtasks,  job  conditions  and  standards  and  the 
supporting  skills,  knowledges,  and  attitudes. 

Sample  Task  Identification  Matrix. 

SLIDE  4  (Appendix  H) 

This  slide  shows  a  sample  extract  of  the  matrix  we  are  using.  The 
tasks  as  mentioned  before  are  identified  as  either  fundamental  -  that 
is  common  to  all  eight  MOS,  branch  or  MOS  specific.  A  separate 
matrix  for  each  MOS  is  identified  with  duty  positions  for  each  skill 
level  listed.  An  "X"  is  placed  in  the  matrix  if  the  task  is  performed 
by  the  duty  position.  For  MOS  IIB  we  identified  11  duty  positions, 
some  of  which  are  shown  on  this  sample.  Some  tasks  from  the  "CBR" 
category  are  listed.  Of  course  the  task  list  includes  both  soft  skill 
and  hard  skill  tasks.  Prepare  a  CBR  Plan  is  undoubtedly  a  soft  skill 
task  whereas  the  others  are  closer  to  being  classified  as  hard  skill. 
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Rationale  for  Eight  Combat  MOS  Study. 
SLIDE  5 


RATIONALE  FOR  EIGliT  COMBAT  MOS  STUDY 

Prevent  duplicative  systenns  engineering  efforts 

Prepare  consolidated  training  literature  for  job  incumbents 

Provide  audio-visual  lessons  for  continuation  training  in  units 

Provide  diagnostic  tests  for  individual  skills 

Provide  "how-to-train"  literature  for  unit  leaders 

Provide  study  literature  and  lessons  for  MOS  tests 

1.  Sk3me  o?  the  reasons  for  developing  this  study  are  shown  here. 

By  doing  this  exercise  together  it  was  felt  th.t  it  would  prevent  each 
school  from  systems  engineering  all  the  subjects  areas.  Many  schools 
of  course  had  already  done  extensive  systems  engineering  work  on  many 
of  the  subject  areas  in  designing  the  NCOES  resident  courses,  however 
many  tasks  involved  in  this  study  involved  the  10  and  20  skill  levels. 

2.  One  of  the  big  reasons  for  the  study  was  to  make  it  easier  for 
a  soldier  to  study  for  his  MOS  test.  Currently,  for  example,  the 
infantry  soldier  has  to  study  50  or  so  field  manuals  and  other  references. 
This  study  is  intended  to  reduce  these.  The  initial  plan  was  to  have 

3  manuals  to  include  a  fundamentals  manual,  a  career  management 
field  or  branch  manual,  and  an  MOS  specific  manual.  However,  we 
may  end  up  with  more  than  three.  Nevertheless  in  any  event,  the  number 
of  study  references  will  be  greatly  reduced. 

3.  The  major  product  of  the  study  will  be  audio-visual  lessons 
produced  by  each  combat  arms  school  and  provided  to  units  throughout 
the  Army.  Performance  oriented  lessons  will  be  available  at  battalion 
level. 

4.  Also  planned  is  the  production  of  diagnostic  tests  from  the  tasks 
identified.  These  tests  will  be  available  to  the  unit  commander.  As 
new  men  arrive  they  will  take  these  tests  to  determine  their  status  of 
training.  The  man  can  then  be  designated  to  take  certain  audio-visual 
lessons  in  areas  needed. 

5.  Some  tasks  are  not  adaptable  to  the  audio-visual  methods. 

Many  of  these  will  be  supervisor's  tasks.  How-to-train  publications  and 
lessons  will  be  developed  for  these  areas. 

11-47 


6,  Finally,  when  these  critical  tasks  identified  are  incorporated 
into  the  MOS  tests,  the  training  publications  and  audio-visual  lessons 
developed  will  provide  the  study  references. 

Problem  Areas  in  Eight  Combat  MOS  Study.  This  slide  shows  some 
of  the  problem  areas  experienced  so  far  in  the  project  with  soft  skill 
tasks. 

SLIDE  6 

SIGNIFICANT  PROBLEMS  IN  SYSTEMS  ENGINEERING 
SOFT  SKILL  TASKS 

Determining  proper  level  of  specificity  for  task  statements 
Determining  difference  in  standards  and  supporting  SKAS 
between  leaders  and  their  subordinates 
Providing  for  cross-training  requirements 
Stating  preventive  tasks  eg  "avoid  improper  use  of  drugs" 
in  positive  terms 

Determining  a  measurable  job  standard  for  soft  skill  tasks 

Writing  the  task  statements  at  the  proper  level  is  a  continuing  problem 
area.  With  soft  skill  tasks,  my  experience  indicates  that  it  is  very 
important  to  break  down  the  tasks  to  specific  low  level  requirements. 

It  is  only  then  that  a  measurable  job  standard  can  be  derived.  For 
example.  Prepare  a  Unit  CBR  Plan,  is  too  broad  a  task  to  develop  a 
I  meaningful  standard.  Even  the  subtasks  of  Prepare  a  Chemical 

Defense  Plan  or  Prepare  a  Biological  Defense  Plan  are  too  broad. 
Perhaps  a  subtask  or  task  like  Select  Personnel  for  the  Monitoring  Team 
is  more  correct  as  the  level  of  specificity.  The  standards  may  differ 
for  the  same  task  between  the  doer  and  the  supervisor.  This  has 
presented  some  problems.  The  standards  for  the  supervisor  would 
include  different  considerations  for  example,  who,  what,  when,  how, 
and  why.  In  many  cases  additional  skills,  knowledges,  and  attitudes 
may  also  be  required  for  supervisory  tasks.  Another  problem  area 
is  cross-training.  It  is  easy  to  say  that  we  want  everyone  to  read  a 
map,  use  a  compass,  operate  a  radio  or  service  a  vehicle.  But  is 
this  a  cross-training  requirement  or  does  he  actually  do  it  in  his  duty 
position?  For  the  purpose  of  the  eight  combat  MOS  study  we  checked 
the  matrix  for  tasks  performed  only  by  the  incumbent  in  the  duty 
position.  The  unit  commander  is  responsible  for  cross-training  and 
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he  can  use  the  task  inventory  and  determine  his  cross-training  require¬ 
ments.  How  do  you  state  as  tasks,  "don't  steal,  don't  gamble,  don't 
use  drugs,  don't  go  AWOL  or  don't  fight  with  your  buddies"  ?  Also, 
how  do  you  write  a  job  standard  for  negative  tasks  such  as  these?  My 
answer  is:  don't  try,  they  shouldn't  be  tasks  to  start  with.  They  are 
attitudes  or  knowledges  supportmg  the  tasks  Preserve  your  Character 
or  Preserve  your  Ability  to  Perform  your  Job.  I'm  sure  all  of  us  are 
concerned  with  the  last  item  in  this  list.  How  do  you  derive  a  job  standard 
for  a  soft  skill  task?  It  is  not  easy,  but  it  can  be  done  and  of  course 
it  must  be  done  if  we  are  going  tc  provide  the  proper  amount  of  training. 

SLIDE  7 

DERIVING  JOB  STANDARDS  FOR  SOFT  SKILL  TASKS 

Define  the  desired  product  resulting  from  task  performance 
Analyze  consequences  that  might  result  from  improper  performance 
or  failure  to  perform  the  task 

Break  down  broad  soft  skill  task  statements  into  subtasks  that 
can  be  measured  in  specific  terms 
Derive  check  lists  for  supervisory  tasks  performed  by  leaders 
Describe  limits  of  acceptability  in  terms  of  speed,  quantity  and 
quality  of  product/work 

Deriving  Job  Standards  for  Soft  Skill  Tasks.  This  slide  shows  the 
procedure  employed  by  the  Infantry  School  in  deriving  job  standards  for 
soft  skill  tasks  in  the  Eight  Combat  MOS  Study.  Many  soft  skill  tasks 
result  in  a  product.  In  such  cases,  parameters  should  be  defined. 

In  other  cases,  analyze  what  would  happen  if  the  task  was  not  performed 
at  all,  or  what  the  consequences  would  be  if  an  inexperienced  or  unskilled 
job  incumbent  performed  it.  How  would  the  unskilled  incumbent  go 
astray?  How  would  it  affect  the  unit's  mission?  Having  made  this 
analysis  then  the  standard  can  consist  of  those  things  he  must  do  to 
avoid  the  consequences  of  poor  performance.  The  further  you  break¬ 
down  soft  skill  tasks  the  easier  it  is  to  derive  the  standard.  For 
supervisory  tasks  or  unit  tasks,  a  series  of  detailed  check  lists  like  the 
check  lists  for  a  unit  ATT  can  be  the  standard.  As  with  hard  skill 
tasks,  limits  can  often  be  described  in  terms  of  speed,  quantity  or 
qxiality. 
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Smnmary.  I  hope  I  have  in  thia  short  period  given  you  an  idea  of  how 
we  :  re  systems  engineering  soft  skills  in  the  Eight  Combat  MOS  Study* 
I  feel  it  is  an  exciting  study  and  j.t  will  have  a  tremendous  affect  on 
all  training  conducted  in  units. 


m 


THE  "IDEALS"  SYSTEM  APPROACH 


Or.  Gerald  Nadler 
University  of  Wisconsin 
Madison,  Wisconsin  53706 
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TV  RECORDING  OF  OR.  NAIX.ER'S  PRESENTATION  IS 

AVAILABLE  UPON  REQUEST  FROM  CONARC  TRAINING 

AIDS  AGENCY,  FORT  EUSTIS,  VIRGINIA  23604. 

(A  60  minute  blank  video  caasetce  tape  must 
be  provided) 


WORKING  GROUP  DISCUSSIONS 
SUMMARY 


The  organisation  of  conferees  into  small  working  groups  and  the 
scheduling  of  presentors  to  appear  in  each  group  for  questioning 
and  discussion  stimulated  extensive  interaction  among  participants. 

The  nature  of  the  discussions  indicated  that  conferees  were  experi¬ 
enced  and  knowledgeable  in  the  area  of  systems  engineering  and  eager 
to  exchange  ideas  on  problems  associated  with  systems  engineering  of 
soft-skill  courses.  The  discussions  also  revealed  that  schools  with 
courses  containing  considerable  amounts  of  soft-skill  content  had 
much  greater  difficulty  applying  CON  Reg  350-100-1  than  schools 
with  courses  containing  considerable  amounts  of  hard-skill  content. 

Initially,  conferees  were  inclined  to  employ  a  hard  skill  -  soft  skill 
dichotomy  of  tasks  as  a  referent  for  discussions.  However,  it  became 
apparent  early  in  the  discussions  that  a  distinction  was  unrealistic. 

Not  only  was  a  working  definition  of  soft  skills  virtually  impossible  to 
develop  but  the  terms  hard  skill  and  soft  skill,  as  defined  in  CON  Reg 
350-100-1,  did  not  represent  mutually  exclusive  categories  of  course 
relevant  tasks.  Furthermore,  it  became  apparent  that  such  a  categor¬ 
ization  of  tasks  is  not  essential  to  the  systems  engineering  process  and, 
in  some  instances,  may  even  generate  misunderstandings.  As  a  result, 
a  majority  of  the  groups  con<~’luded  that  no  distinction  should  be  made 
between  hard  skills  and  soft  skills  and  recommended  that  the  term 
"soft  skills"  be  eliminated  from  systems  engineering  terminology. 

During  the  discussion  on  task  classification  it  was  proposed  in  several 
groups  that  tasks  are  of  such  nature  that  they  can  be  distributed  along 
a  continuum  with  specific,  easily  described  tasks  on  one  end  and  general, 
difficult-to-describe  tasks  on  the  other.  In  discussing  this  concept, 
conferees  addressed  the  question  --  is  CON  Reg  350-100-1  suitable  for 
all  tasks  on  the  continuum?  Some  conferees  were  adamauit  in  the  opinion 
that  the  regulation  is  appropriate  and  adequate  for  all  tasks,  while  others 
were  equally  adamant  that  the  regulation  is  not  suitable  for  the  general 
tasks.  A  considerable  number  appeared  to  hold  opinions  ranging  be¬ 
tween  the  two  extremes,  believing  the  regulation  could  apply  reasonably 
well  to  most  tasks  and  functions  if  flexibility  were  permitted  in  the 
degree  of  specificity  required  for  objectives  and  other  systems  engineer¬ 
ing  products  in  the  general  task  areas. 
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In  other  group  discussions  a  variation  of  the  task  continuum  concept 
was  proposed.  In  this  concept»  specific  tasks  were  characterized  as 
being  composed  primarily  of  specific  information  and  psychomotor 
skills^  while  general  tasks  were  considered  to  consist  of  process  with 
an  under  girding  of  specific  knowledge  and  mental  skills.  In  this  regard 
it  was  noted  that  decision  making  is  a  process,  but  in  order  to  make  a 
sound  decision  an  individual  mu8\:  possess  the  fundamental  facts 
relevant  to  the  decision.  It  was  also  noted  that  the  mental  skills  of 
discrimination  and  reasoning  are  a  part  of  the  decision  making  process. 
One  group  stressed  the  importance  of  process  in  general  tasks  by 
concluding  that  in  systems  engineering  general  tasks  we  most  definitely 
need  to  be  more  concerned  with  "how  a  decision  is  made"  and  "why  it 
is  made"  than  with  the  content  or  the  nature  of  the  decision  itself. 

Reacting  to  the  various  approaches  to  systems  engineering  of  soft  skills 
which  had  been  presented  to  them,  conferees  expressed  concern  about 
the  presence  of  multiple  action  elements  in  sample  objectives  and  the 
absence  of  identifiable  standards  in  several  of  the  approaches.  The 
Behavioral  Model  approach  presented  by  Dr.  Whitmore  was  considered 
to  be  viable,  and  the  feature  of  analyzing  some  performances  from  a 
task  standpoint  and  others  from  the  standpoint  of  process  was  well 
received.  Goal  Analysis  presented  by  Dr.  Mager  and  the  IDEALS 
systems  approach  presented  by  Dr.  Nadler  were  considered  relevant 
and  promising,  but  in  need  of  further  definition  and  explication  before 
they  could  be  applied  to  the  specific  training  situations  of  CONARC 
schools.  The  philosophical  approach  which  gave  consideration  to 
cybernetic  principles  and  recognized  process  as  an  important  element 
in  course  structure  was  very  provocative.  Many  conferees  were 
intrigued  by  this  approach  and  by  its  possible  use  in  such  functional 
and  general  task  areas  as  supervision,  leadership  and  counseling. 
Consequently,  a  majority  of  the  groups  recommended  that  CON  Reg 
350-100^1  be  revised  to  include  a  cybernetic  or  comparable  approach 
for  placing  proper  emphasis  upon  the  process  aspects  of  content  in 
CONARC  courses.  In  connection  with  this  revision,  conferees  expressed 
a  desire  for  an  annex  containing  examples  of  systems  engineering 
products  in  the  general  task  areas.  These  examples  would  include 
tasks,  objectives,  conditions,  and  standards  as  well  as  evaluative 
instruments  and  techniques. 
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While  developing  primary  recommendations  in  accordance  with  the 
purpose  of  the  discussions,  working  groups  developed  some  subsidiary 
findings  and  recommendations  as  well.  Some  of  these  are  listed  below. 

1.  Schools  should  insure  tha.t  group  interaction  and  group  process 
are  not  lost  as  a  result  of  the  emphasis  placed  on  individual  job  skills 
and  individualized  instruction. 

2.  Schools  having  common  systems  engineering  problems  should 
use  a  commonality  approach  to  resolve  these  problems  (e.  g.  Eight 
Combat  Arms  MOS  Study). 

3.  CONARC  should  establish  a  course  development  workshop  that 
will  provide  training  assistance  and  experience  in  systems  engineering 
the  broad,  conceptual,  general  tasks. 

4.  Schools  need  a  regulation  that  is  general  enough  in  nature  to 
meet  the  needs  of  all  schools  for  mission  guidance,  yet  not  unduly 
restrictive  or  limiting  to  any  school  in  determining  how  it  can  best 
achieve  its  purpose. 

5.  Schools  desiring  an  exception  to  a  systems  engineering  require¬ 
ment  should  submit  a  request  documenting  the  desired  change(s),  the 
rationale  for  the  desired  change(3),  and  the  procedures  that  will  be 
followed.  Prior  to  approval /disapproval  CONARC  should  review  the 
request  for  possible  research  projects,  need  for  expert  assistance 

and  commonality  of  problems  among  schools. 

6.  Systems  engineering  should  not  be  considered  mandatory  for 
all  schools  and  courses. 

7.  CONARC  should  provide  assistance  with  regard  to  guidance 
and  resources  for  both  internal  and  field  validation  of  instructional 
technology  (systems  engineering). 
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APPENDIX  C 


A  BEHAVIORAL  MODEL  (PURPOSE  ANALYSIS)  OF  A  LEADERSHIP  FUNCTION 
BASED  ON  PRINCIPLES  OF  BEHAVIOR  MODIFICATION 

01  Designs  and  Implements  management  practices  which  strengthen 
productive  behaviors  and  weaken  counter-productive  behaviors 
emitted  by  his  people, 

01  01  Designs  and  implements  practices  for  reinforcing  productive 

behavior . 

01  01  01  Identifies  and  defines  productive  behaviors. 

01  01  02  Specifies  and  implements  techniques  for  monitoring 

the  occurrence  of  productive  behaviors. 

01  01  03  Identifies  potentially  effective  and  feasible  positive 

reinforcers. 

01  01  04  Designs  and  implements  techniques  for  the  contingent 

administration  of  positive  reinforcers  on  an  appropriate 
schedule • 

01  01  05  Modifies  reinforcement  program  if  desired  changes  fail 

to  occur. 

01  02  Designs  and  implements  practices  for  minimizing  the  inadvertent 

reinforcement  of  counter-productive  or  disruptive  behavior, 

01  02  01  Identifies  and  defines  counter-productive  or  disruptive 

behaviors « 

01  02  02  Specifies  and  implements  techniques  for  monitoring  the 

occurrence  of  counter-productive  or  disruptive  behaviors. 

01  02  03  Designs  and  implements  techniques  for  eliminating  the 

inadvertant  reinforcers. 

01  02  04  Modifies  reinforcement  elimination  program  if  desired 

changes  fail  to  occur. 

01  03  Designs  and  implements  practices  for  minimizing  the  occurrence 

of  aversive  stimuli  in  the  environment. 

01  03  01  Identifies  aversive  stimuli  in  the  environment. 

01  03  02  Designs  and  implements  techniques  for  eliminating  or 

reducing  aversive  stimuli  in  the  environment. 

Ix-^0 


GENERAL  OPERATIONAL  CHARACTERISTICS 
OF  A  "MOTIVATED  WORKER" 


01  Performs  an  adequate  amount  of  work  that  leads  to  the  attainment 
of  organizational  goals. 

02  Exhibits  a  low  frequency  of  interference  with  the  work  of  others. 

03  Coordinates  his  work  activities  with  the  work  activities  of  others 
in  keeping  with  organizational  goals. 

04  Helps  others  to  perform  their  work,  if  necessary  to  meet  organiza¬ 
tional  goals  and  if  he  is  able  to  do  so. 

05  Minimizes  the  effects  of  outside  disruptions  on  his  own  work  and 
that  of  others. 

06  Chooses  to  remain  in  the  organization  for  the  remainder  of  his 
work  career. 


SUGGESTED  READINGS 


Goal  Analysis  by  Robert  F.  Mager,  Fearon  Publishers/Lear  Siegler, 
Inc.,  1972. 

Behavioral  Technology:  Motivation  and  Contingency  Management 
(Volumes  1  and  II,  and  a  student  manual)  by  Lloyd  Homme  and 
Donald  Toste,  Individual  Learning  Systems,  P.O.Box  3388,  San 
Rafael,  California  94902. 

Teaching:  A  Course  in  Applied  Psychologytby  Wesley  C.  Becker, 
Siegfried  Engleman,  and  Don  R.  Thomas,  Science  Research 
Associates,  Inc.,  Palo  Alto,  California,  1971. 

Managing  Behavior:  Parts  J,  II,  and  III  by  Vance  R.  Hall,  H  &  H 
Engerprises,  Inc.,  Merriam,  Kansas,  1970. 
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WELCOME  TO  THE  METHODS  AND  MEDIA  WORKSHOP 

Welcome  to  the  Methods  and  Media  Workshop  of  the  Soft  Skill  Training 
Conference.  This  session  will  consist  of  five  presentations.  Four 
^f  the  presentations  will  be  made  by  representatives  of  four  of  the 
service  schools.  They  will  discuss,  their  experience  in  the  select 
tion  and  application  of  various  methods  and  media  to  some  of  their 
training  problems.  Then  Mr.  Moon  from  the  HumRRO  Unit  at  Fort  Knox 
will  give  us  a  brief  report  on  Work  Unit  SMMART. 

I  will  serve  as  Chairman  and  Introducer  of  the  speakers.  Each 
speaker  will  have  approximately  35  minutes  for  his  presentation  and 
discussion.  We  want  active  participation  by  the  conferees  during 
the  discussion  periods.  Your  contributions  are  needed  so  that  we 
can  give  adequate  consideration  to  the  many  possibilities  of  Methods 
and  Media.  Any  questions  not  adequately  answered  during  the  speakers' 
allotted  30  minutes  can  be  considered  during  the  final  discussion  and 
sumnary  period. 

I  will  now  introduce  our  six  speakers  in  the  order  of  their  appear* 
ance  on  the  program: 

LTC  James  H.  Sewell,  Career  Course  Manager  at  the  US  Army  Chemical 
Center  and  School 

Dr.  Charles  W.  Kroen,  Research  Psychologist  at  the  US  Army  Signal 
Center  and  School 

Mr.  Clarence  C.  Newsom,  Educational  Advisor  at  the  US  Army  Aviation 
School 

Major  Bernard  Meacham,  Chief,  Automation  Branch,  Supply  Career 
Department,  US  Army  Quartermaster  School 

Mr.  Harold  L.  Moon,  Research  Psychologist  with  the  HumRRO  Unit  at 
Fort  Knox 

Since  we  do  have  a  full  program  we  will  move  right  on  to  our  first 
presentation,  Methods  and  Media  Used  in  the  Redesigned  Chemical 
Officer  Advanced  Course,  by  LTC  Sewell! 
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CONARC  SOFT  SKILL  TRAINING  CONFERENCE 
Methods  and  Media  Used  in  the 
Redesigned  Chemical  Officer 
Advanced  Course 

Lie  James  H.  Sewell 
USA  Chemical  Center  and  School 
Ft  McClellan,  AL 

The  title  of  this  presentation  is  centered  around  Methods  and  Media. 

I  am  not  quite  sure  that  I  know  what  is  meant  or  understood  by  these 
two  terms.  The  Oxford  Dictionary  tells  me  that  media  are  vehicles 
of  perception  and  that  a  method  is  a  special  form  or  procedure  used 
in  any  mental  activity.  But  dictionaries  are  historical  documents 
more  than  they  are  current  authorities.  They  can  tell  me  only  what 
usage  was  made  of  terms  by  past  writers,  not  what  my  present  meaning 
is  or,  more  importantly,  what  meaning  will  be  given  the  terms  by  my 
present  listeners.  We  are  all  engaged  in  a  process  of  approximation. 
I  will  say  things  that  approximate  what  is  in  my  mind.  The  thoughts 
that  occur  in  your  minds  will  approximate  what  I  tried  to  say.  I 
ask  of  you  only  that  you  do  not  attach  undue  demands  on  the  words  I 
use,  for  they  are  our  tools  of  approximation.  What  is  in  my  mind 
at  this  moment  is  different  from  that  when  I  was  putting  together 
these  notes.. .yet  I  am  reasonably  satisfied  that  the  words  express 
some  of  my  present  feelings. 

I  am  uncomfortable  right  now.  X  have  been  uncomfortable  for  a  long 
time.  Certainly,  this  is  a  group  of  prestige  and  influence  enough 
to  make  me  humble.  But  that  is  not  all  of  it.  We  are  all  person¬ 
ally  involved  with  the  evolution. . .or  revolution. . .that  is  taking 
place  in  the  Army  School  System.  And  that  does  generate  genuine 
distress  and  anxiety  in  me,  for  there  is  no  more  crucial  institution 
in  the  Army  than  its  school  system.  At  one  time  or  another,  every 
soldier •• .man  or  woman.. .will  be  influenced  by  an  Army  school.  I 
cannot  think  of  anothei^  Army  institution  that  can  make  that  claim. 

We  alone  will  see  them  all.  And  we  can  be  certain  of  it.  What  will 
we  do  with  this  opportunity? 

There  was  a  story  told  of  a  self-help  advisor  working  in  a  depressed 
rural  conmunity .  He  asked  a  woodcutter  how  much  wood  he  was  able  to 
cut  in  a  week  with  his  single-bitted  axe.  The  woodcutter  answered 
that  he  usually  got  about  three  cords.  The  advisor  then  gave  him 
a  new  McCulloch  chain  saw  and  told  him  he  would  check  back  in  a  few 
weeks  to  see  what  improvement  the  saw  would  make.  Upon  questioning 
the  woodcutter  several  weeks  later,  he  was  told  that  only  one  cord 
per  week  had  been  cut.  The  advisor  felt  that  there  might  be  some¬ 
thing  wrong  with  the  saw.  He  primed  it,  gave  the  cord  a  pull,  and 
was  rewarded  with  a  full-throated  roar  as  the  engine  caught.  The 
woodcutter  cried,  "What's  that  noise  I" 
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I  wonder,  now,  if  we  are  not  much  like  the  woodcutter:  swinging  our 
new  tools  like  an  axe.  We  have  a  tool  of  vast  potential  in  the  con¬ 
cept  of  System  Engineering, . .and  we  use  it  to  produce  the  same  type 
of  ineffective  curricula  that  we  used  to  get  on  intuition  alone. 

have  a  new  type  of  soldier,  and  we  appeal  to  him  or  her  with  the 
myths  of  two  generations  ago.  I  recently  observed  a  class  in  which 
the  motivational  quote  used  was  17  years  old!  We  have  educational 
aids  such  as  ETV,  computer  assisted  instruction,  and  sophisticated 
audio-visual  techniques  which  we  use  to  give  answers  to  questions 
that  will  never  be  asked  1  One  of  our  faculty  came  up  with  this 
limmerick: 


An  Advanced  Course  student  with  tact 
Absorbed  many  answers  he  lacked 
But  out  on  the  job 
He  decried  with  a  sob 
How  does  one  fit  answer  to  fact? 

And  yet  one  must  notice  that  the  Army  continues  to  produce  highly 
competent  people  from  its  school  system.  Is  it  perhaps  that  we 
produce  talented  and  capable  people  despite  our  system?  And  has  it 
occurred  to  you  that  our  Chief  of  Staff  twenty  years  from  now  has 
already  finished  his  Advanced  Course?  Of  course,  he  still  will 
have  a  chance  to  be  rounded  out  by  Command  and  General  Staff  School 
and  the  War  College;  but  must  we  rely  overmuch  on  these?  For  the 
great  number  of  officers  who  will  attend  neither  of  these  senior 
schools,  the  Advanced  Course  is  the  last  they  may  see  of  Army 
schools.  It  would  appear  that  the  Advanced  Course  should  be 
sufficient  preparation  for  the  general  staff  officer. 

But  what  of  methods  and  media?  Consider  for  a  moment  the  store  of 
knowledge,  skills,  attitudes,  behavior,  or  "know  how"  that  you  have 
accrued  since  you  have  been  associated  with  the  Army.  Although 
certainly  not  complete  (for  1  know  of  no  manager  who  is  absolutely 
satisfied  with  his  performance  on  all  tasks) ,  you  do  owe  whatever 
effectiveness  you  have  to  these  abilities.  How  much... what  pro¬ 
portion... of  your  present  store  of  knowledge  was  obtained  as  a 
student  in  a  classroom?  Would  anyone  guess  more  than  107o?  How 
about  5%?  No  takers?  For  myself,  I  must  say  that  something  less 
than  1%  of  my  ability  was  acquired  in  a  classroom  as  a  student. 

But  we  do  learn,  do  we  not?  Have  you  not  learned  quite  a  bit, 
some  of  it  being  of  real  value?  How  did  you  learn  it? 

In  real  life,  we  learn  things  by  facing  a  task  or  being  involved  in 
a  situation,  and  striving  to  "solve"  the  problem.  We  do  this  by 
talking  and  interacting  with  people  or  by  independent  study.  We 
are  motivated  by  our  perception  of  people  whom  we  odmire.  We  try 
to  emulate  them,  to  perform  our  jobs  in  a  creditable  fashion.  Most 
work  groups  of  people  who  really  contribute  to  one  another  are 
relatively  small:  4  or  5  people.  If  the  group  becomes  much  larger, 
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then  too  much  effort  will  be  spent  in  posturing  before  the  group 
It *8  hard  to  fake  it  with  a  small  group  for  very  long. 


There  can  be  no  discussion  of  the  media  used  at  the  Chemical  School 
without  a  discussion  of  the  method.  In  a  real  sense,  the  medium  is 
the  method.  Let  me  describe  how  our  course  came  about  and  what  it 
is  based  on.  In  this  way,  we  will  see  the  effect  of  a  new  concept 
of  what  a  student  is  on  the  design  of  a  career  course  and  on  the 
educational  institution  itself. 

In  the  Spring  of  1971,  a  design  group  was  given  the  broad  charter 
of  "taking  a  look  at  the  Advanced  Course."  We  began  with  the  normal 
system  engineering  step  of  defining  the  tasks  a  graduate  would  be 
expected  to  be  able  to  accomplish.  When  you  consider  that  the  gradu¬ 
ate  we  were  talking  about  was  an  Advanced  Course  Officer. ..a  captain 
with  3-5  years  service. • .the  list  was  very  imposing.  In  fact,  the 
partial  list  of  tasks  covered  the  walls  and  ceiling  of  the  room  and 
was  far  from  complete.  It  was  not  just  that  there  were  so  many  tasks 
that  could  conceivably  fall  to  this  type  of  officer,  but  it  was 
apparent  that  no  human  alive  could  master  all  of  them.  Were  we  to 
pretend  that  we  would  attempt  to  produce  a  graduate  having  skills 
never  before  mastered?  Or  would  we  take  a  more  realistic  view  and 
attempt  to  define  a  more  meaningful  set  of  goals? 

The  decision  was  made  to  completely  ignore  all  existing  goals  and 
constraints  in  the  design  of  the  new  course.  We  would  begin  with 
the  bare  minimum  of  assumptions  and  seriously  question  any  of  these 
that  began  to  take  on  any  amount  of  importance.  In  a  sense,  we 
"buried  the  sacred  cow. • .alive."  We  assumed  chat  the  course  would 
last  36  weeks.  Certainly  this  was  a  safe  assumption.  Beyond  that 
we  assumed  only  that  the  student  we  would  have  would  be  a  captain  or 
senior  lieutenant  with  3  to  3  years  of  service.  This  assumption 
proved  to  be  safe  and  did  not  constrain  us  later.  As  you  can  see, 
our  solution  grid  was  very  wide. 

Rather  than  asking  what  tasks  a  graduate  must  perform,  we  asked 
what  kind  of  a  person  could  respond  to  the  demands  exemplified  by 
the  tasks.  Rather  than  performance  objectives  in  terms  of  tasks  we 
arrived  at  a  profile  of  the  graduate  in  terms  of  his  human  qualities. 
The  profile  looked  like  this: 

@  To  be  technically  proficient 

@  To  be  able  to  identify  and  analyze  complex  problems 

@  To  think  Imaginatively  and  in  broad  range 

&  To  be  skilled  in  interpersonal  relations 
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@  To  be  able  to  communicate  complex  issues  clearly  in  speech 
and  writing 

0  To  be  persuasive 

@  To  be  aware  of  the  major  issues  and  problems  of  the  Army  and 
Society 

(9  To  be  relaxed  in  the  presence  of  high  ranking  officers  and 
officials 

(?  To  be  familiar  with  problems  and  procedures  at  various 
levels  of  command  ‘ 

We  felt  that  this  profile  defined  the  main  dimensions  of  an  effective 
officer.  These  are  the  skills  that  get  jobs  done.  We  cried  the 
profile  out  on  everyone  we  could  corner.  The  responses  fell  into 
two  groups.  First,  people  said  they  were  ^'motherhood"  items... 
intrinsically  good.  We  disagreed  with  this  observation  on  two 
grounds:  that  the  items  were  essential  for  effectiveness  and  that 
they  could  be  influenced  during  the  Advanced  Course.  The  second  set 
of  responses  indicated  that  we  were  "right  on."  Curiously,  general 
officers  fell  in,o  this  latter  group.  They  said  we  were  describing 
precisely  the  things  that  made  them  as  effective  as  they  were. 

It  is  important  to  note  that  this  profile  described  the  whole  man 
and  not  just  his  working  dimensions.  Unfortunately,  in  the  eyes  of 
some,  we  must  contend  with  the  whole  man  on  a  job.  We  cannot  have 
only  his  job  functions  without  also  getting  his  personality,  his 
personal  problems,  hang-ups,  and  all  his  human  attributes. 

Having  accepted  this  profile  as  our  goal,  we  decided  that  student- 
centered  learning  was  the  only  method  that  would  address  the  whole 
man.  If  we  were  to  influence  the  complexity  of  each  student,  we 
must  involve  his  total  being  in  the  learning  process.  We  had  then, 
and  have  now,  an  unfailing  respect  for  the  ability  and  responsibil¬ 
ity  that  resides  in  the  student.  All  learning  must  be  done  by  the 
student  and  he  is  the  only  one  who  can  motivate  himself.  We,  as 
faculty,  can  only  do  what  we  can  to  make  it  happen. 

Given  that  the  student  would  provide  the  horsepower  to  learn,  we 
elected  to  set  the  stage  by  way  of  real-life  situations  that  the 
student  could  easily  imagine  himself  in.  We  gleaned  the  inboxes  of 
units  everywhere  for  the  types  of  problems  that  typify  daily  work. 

For  the  most  part,  dates  and  names  were  blocked  out  of  on-going 
actions  and  the  problems  were  fed  to  the  student.  This  is  the  way 
we  are  tasked  in  real-life,  and  it  worked  for  our  students. 


It  has  been  said  that  experience  is  the  best  teacher,  bat  that  the 
tuition  fees  are  high.  We  worked  to  lower  the  cost  to  the  student. 
Examinations  were  eliminated.  Class  rankings  were  eliminated.  The 
atmosphere  was  made  as  threat-free  as  we  could  make  it.  Since  many 
of  the  problems  passed  out  were  still  being  worked  on  in  real  life, 
there  was  no  need  for  school  solutions.  Indeed,  without  the 
necessity  to  grade  the  student,  ve  found  no  need  for  solutions  other 
than  those  arrived  at  by  the  students. 

The  effect  of  real  learning  is  that  the  person  changes  because  of 
it.  If  a  person  learns  from  solving  a  problem  or  coping  with  a 
situation,  it  is  certain  that  he  would  work  that  problem  differently 
in  the  future.  Mistakes  would  be  avoided  and  successful  methods 
would  be  intensified.  For  this  reason,  it  was  essential  that  the 
problem  solving  process  be  undertaken  again  and  again.  Our  course 
was  therefore  structured  in  an  iterative  fashion.  We  elected,  quite 
arbitrarily,  to  have  six  phases  of  about  equal  length  in  the  nine- 
month  course.  One  phase,  the  Radiological  Safety  Phase,  would  be 
treated  traditionally  because  of  its  quantitative  and  technical 
nature.  The  other  phases  were  titled  Company /Battalion,  Brigade, 
Division,  CONUS  Installation,  and  Logistics /Higher  Headquarters, 

The  titles  merely  suggest  the  level  of  problems  addressed.  Any 
other  hierarchy  that  would  maintain  the  repetitive  and  increasing 
complexity  of  problems  would  suffice.  The  idea  was  to  expose  the 
student  to  a  problem-solving  exercise  again  and  again. 

One  further  thing  is  pertinent  regarding  these  problems.  The  prob¬ 
lem  itself  was  imbedded  in  a  situation.  It  was  up  to  the  student 
to  identify  that  problem.  He  must  find  it,  and  formulate  it  for 
himself,  then  must  invent  a  range  of  solutions.  The  single  greatest 
cause  of  ineffective  solutions  in  real  life  is  that  the  wrong  prob¬ 
lem  is  addressed  or  that  it  is  improperly  formulated. 

The  normal  mode  for  class  work  was  the  small  group.  Students  were 
grouped  into  4-5-man  groups,  each  group  being  assisted  by  a  Faculty 
Consultant.  Let  me  point  out  that  the  Faculty  Consultant  was  in  no 
way  a  subject-matter  expert  on  the  material  addressed  by  the  student. 
His  role  was  to  encourage  and  stimulate  the  student  in  a  direct, 
personal  way.  His  normal  method  was  face-to-face  confrontation  and 
penetrating  questions.  All  Faculty  Consultants  are  Field  Grade 
officers.  In  operational  control  of  the  phase  is  the  Phase  Leader, 
also  a  Field  Grade  officer.  He  is  responsible  for  planning  the 
phase,  for  designing  the  situations  to  be  presented,  and  for  meter¬ 
ing  the  flow  of  situations  to  the  student.  He  has  full  authority 
to  change  the  schedule,  to  speed  it  up  or  slow  it  down.  In  short, 
he  is  the  man  on  the  ground  with  the  detailed  knowledge  and  he  is  in 
full  control. 


There  are  no  set  class  hours  for  the  student.  He  is  given  the  situ¬ 
ation  and  the  relevant  suspense  dates.  It  is  his  task  to  organize 
the  work  so  that  the  suspense  is  met  with  an  effective  and  practical 
solution.  He  may  request  assistance  from  the  full  faculty  on  any 
S’ bject  by  going  through  his  Consultant.  Traditional  platform 
instruction  does  occur,  but  the  student  has  the  option  of  rejecting 
it  if  he  so  desires.  In  other  words,  relevance  to  the  student  is 
the  criterion,  not  that  the  faculty  thinks  it  is  **good  for  him.” 

It  is  appropriate  to  mention  that  in  handling  three  classes  under 
this  method,  there  has  not  been  a  single  instance  of  the  student 
taking  advantage  of  the  system  for  his  own  selfish  needs.  The  stu¬ 
dent  has  always  demonstrated  more  dedication  and  sense  of  responsibility 
than  ever  in  my  experience. 

Students  are  evaluated  continuously  on  an  informal  basis,  and  after 
the  end  of  each  phase  in  a  formal  fashion.  At  the  close  of  the  phase, 
the  Faculty  Consultants  and  Phase  Leaders  meet  to  discuss  each 
individual  student.  The  orientation  is  towards  needs  each  student 
has  and  how  they  may  be  satisfied  in  future  phases*  The  direction  is 
towards  changing  the  course  to  fit  the  student  rather  than  a  ranking 
of  students.  During  the  course  each  student  is  carefully  evaluated 
by  a  minimum  of  14  field  grade  officers. 

Our  experience  so  far  has  been  that  there  is  no  difficulty  in  select¬ 
ing  the  Distinguished  Graduate.  Ke  stands  head  and  shoulders  above 
the  class  and  is  obvious  to  everybody.  As  for  Honor  Graduates,  we 
have  surveyed  the  classes  themselves  for  those  students  who  are 
judged  to  be  the  most  likely  to  succeed.  The  students*  judgments 
coincide  perfectly  with  those  of  the  faculty,  so  this  is  not  as  big 
a  problem  as  one  would  imagine.  After  a  long  period  of  time  in 
close,  personal  contact  with  each  student,  valid  evaluations  can  be 
made  • 

So  there  you  have  our  method.  It  is  student-centered,  situation 
stimulated,  and  iterative.  The  medium  is  a  threat-free  atmosphere 
dedicated  to  satisfying  the  needs  of  the  students.  Operational  con¬ 
trol  is  delegated  to  the  Phase  Leader  on  the  spot  rather  than  being 
dominated  by  a  fixed  schedule.  Responsiveness  to  perceived  needs  of 
the  students  is  essential.  Maximum  latitude  is  given  the  student  in 
pursuing  his  goals.  And  the  goals  must  be  those  of  the  student. 

The  results  so  far  have  been  encouraging.  The  students  indorse  the 
course  philosophy  without  qualification.  Their  motivation  continues 
through  the  last  day  of  the  course.  Reports  from  recent  graduates 
state  that  they  consider  the  preparation  given  them  was  relevant 
and  sufficient. 

There  have  been  many  problems.  Foremost  among  them  has  been  the 
agonies  of  our  faculty  in  internalizing  the  philosophy.  No  longer 
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Is  an  Instructor,  or  a  consultant,  or  a  Director  of  Instruction  an 
authority  only  because  he  controls  the  questions  that  nuiy  be  asked 
to  those  he  knows  the  answers  to.  The  student's  questions  are 
generated  by  the  student  and  are  relevant  to  him.  They  are  serious 
questions.  They  are  good  questions  and  they  deserve  being  asked  and 
answered.  But  It  Is  not  the  job  of  the  faculty  to  answer  all 
questions.  Question  asking  and  answer  finding  go  hand  In  hand.  Both 
must  be  done  by  the  student,  and  he  Is  willing  and  able  to  do  them. 

At  one  time,  a  question  that  could  not  be  answered  by  the  subject 
expert  was  considered  to  be  a  "wlse-guy"  question. • .sharpshooting. 
Later  we  qualified  this  so  that  the  Instructor  was  permitted  to 
say,  "I  don't  know  the  answer,  but  I  will  find  out  for  you."  This 
has  the  nice  effect  of  maintaining  the  student  in  a  suoservlent 
role:  and  dependent  on  the  faculty.  Under  our  current  method,  we 
encourage  the  Instructor  to  admit  candidly  that  he  doesn't  know  the 
answer  to  that  question,  but  that  perhaps  it  Is  a  good  question. 
Period.  If  the  student  persists  with  the  question,  we  then  come  to 
a  question  which  Is  appropriate  In  education,  which  Is,  "How  do  you 
think  you  could  go  about  finding  out  the  answer  to  that  question?" 

It  is  not  the  consultant's  role  to  be  the  fount  of  all  knowledge. 

The  student  already  knows  that  no  one  knows  all  the  answers,  so  we 
should  not  worry  too  much  about  giving  ourselves  away.  It  does, 
hoifiever,  put  quite  a  strain  on  us. 


Questions  and  Discussion  following  LTC  Sewell's  Presentation 


Q.  How  do  the  students  react  to  the  faculty  in  this  type  learning 
situation? 

A.  Very  favorably.  The  students  and  faculty  are  closely  associated 
for  a  longer  period  of  time  than  they  were  in  the  former  course.  The 
faculty  member  is  accepted  by  the  students  as  a  resource  from  which 
they  can  receive  guidance.  The  biggest  problem  the  faculty  has  is 
in  learning  to  say,  "I  don't  know.”  Under  the  old  concept  the 
faculty  would  say  in  answer  to  a  question,  "I  don't  know,  but  I'll 
find  out.*'  Under  the  new  concept  the  faculty  is  encouraged  to  say, 

*'I  don't  know.  How  do  you  think  we  ought  to  go  about  finding  the 
answer?*' 

Q.  How  can  you  justify  a  school  system  like  that  where  all  you  are 
doing  is  putting  a  student  in  the  same  realistic  type  situation  he 

would  find  on  the  job?  Why  not  assign  the  student  to  a  variety  of 

real  staff  positions  and  let  him  learn  on  the  job? 

A.  Except  for  one  thing  that  would  be  fine.  On  a  real  staff  you 

cannot  put  the  man  in  a  low  threat  environment.  You  cannot  promise 
him  that  his  boss  won't  chew  him  out.  Also  the  rest  of  the  staff 
is  so  busy  they  won't  have  time  lo  interact  with  the  man  in  a  way 
that  will  be  meaningful  for  learning  through  small  group  inter¬ 
action. 

Q.  How  could  someone  sitting  at  CONARC  justify  a  course  like  this? 

It  seems  that  a  course  of  this  type  could  destroy  the  school  system. 

A,  A  person  sitting  at  CONARC  should  be  called  on  to  justify  only 
the  goals  of  the  course.  The  details  of  attaining  these  goals  can¬ 
not  be  judged  from  a  position  remote  from  the  student;  that  is,  at 
a  position  higher  than  the  School  Commandant.  It  is  true  that  the 
school  system,  as  it  now  exists,  will  have  to  be  changed,  but  that 
is  obviously  needed  anyway. 

Q.  Doesn't  it  take  several  years  to  get  the  various  levels  of 
experience  you  need  in  the  faculcy  to  provide  the  understanding  of 
the  various  situations  you  subject  the  student  to?  ' 

A.  Yes.  The  student  is  subjected  to  far  more  real  si^iuations  at 
the  various  levels  of  command  than  any  faculty  member  has  experi¬ 
enced.  It  is  usually  about  half  way  through  the  course  that  the 
faculty  consultant  will  come  to  the  phase  leader  and  say,  "Could 
you  give  us  an  advance  sheet?**  In  effect  they  are  saying  that  the 
student  has  passed  the  faculty  it.  their  ability  to  solve  the  prob¬ 
lems.  They  already  have  more  exi^erience  than  the  faculty.  The 
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faculty  is  no  longer  the  expert  and  the  students  start  demanding 
less  and  less  of  the  faculty  in  the  way  of  subject-matter  answers. 

The  role  of  the  faculty  shifts  to  that  of  encouraging  and  stimu¬ 
lating  the  students »  and  this  requires  much  training. 

Q.  Would  you  say  that  generally  what  you  have  is  a  catalyst  in 
terms  of  a  problem? 

A.  Yes,  but  we  don't  tell  the  student  what  the  problem  is.  He 
may  have  to  find  the  problem  or  determine  that  there  is  no  problem. 

Q.  Is  your  conventional  media  such  as  TV  and  training  films  used? 

A.  Yes,  conventional  instruction  is  used  as  needed.  In  any  area 
where  the  student  is  weak  and  does  not  possess  the  background  and 
knowledge  to  learn  without  additional  help,  he  has  the  entire 
faculty  as  a  resource.  This  may  come  in  the  form  of  lectures,  TV, 
Training  Films,  or  some  other  method  or  media.  The  students  also 
present  instruction  to  each  other. 

Q.  Three  quick  questions: 

I 

a.  How  many  students  are  in  your  advanced  class? 

b.  How  many  classes  do  you  have  a  year? 

c.  How  do  you  handle  your  electives? 

A.  We  have  27  students  in  the  present  class.  We  had  two  classes 
last  year  and  one  this  year.  We  have  20  or  30  electives.  The 
electives  term  coincides  with  the  terms  at  the  local  Jacksonville 
State  University.  The  students  can  either  take  courses  at  the 
University  or  take  in-house  electives. 

Q.  Do  you  ever  give  the  students  current  school  problems  to  work  on 
such  as  staff  studies  to  solve  current  problems? 

A.  Yes.  A  few  months  ago  they  were  given  the  problem  to  study  the 
Fort  McClellan  Officers  Open  Mess.  It  was  a  well  done  study  and  it 
was  given  to  the  post  commander. 

Q.  What  size  groups  do  the  students  work  in? 

A.  Usually  the  groups  are  small.  It  depends  on  what  they  are  doing. 
Usually  they  work  in  about  a  five-man  group. 

Q.  How  would  you  suggest  that  we  handle  about  a  thousand  Advanced 
Course  students  at  the  Infantry  School  using  this  mode? 
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A.  That *8  a  good  question.  It  has  been  our  experience  that  it 
took  about  7.3  manyears  of  field  grade  officers  to  put  on  one  class. 
I  don't  feel  that  that  is  necessary.  For  the  first  time  the 
faculty  is  not  indispensable.  If  we  mailed  the  situations  to  the 
students  all  we  would  lose  would  be  the  personal  bounce  against  the 
faculty  member.  If  the  student  were  used  to  that  type  learning  he 
could  get  by  without  a  faculty  consultant.  If  we  had  a  super  con¬ 
trol  mechanism  we  could  get  by  without  a  phase  leader,  but  he  is 
our  expansion  joint  that  fixes  everything  to  make  the  day  run. 

During  the  next  phase  we  are  going  to  withdraw  two  of  the  faculty 
consultants.  For  his  five  groups  the  phase  leader  will  have  only 
three  faculty  consultants.  The  phase  leader  of  the  last  phase  said 
that  he  could  get  by  with  only  one  faculty  consultant.  By  that  time 
the  students  have  learned  how  to  learn  on  their  own.  Manpower  con¬ 
sumption  can  be  drastically  reduced.  If  people  are  used  to  it  and 
know  how  it  works,  a  thousand  students  could  be  handled  in  however 
many  groups  you  need.  The  faculty  consultants  are  used  only  about 
two  hours  a  day. 

Q.  Are  there  suspenses  assigned  to  projects? 

A.  Yes,  The  phase  leader  assigns  suspenses  and  he  may  adjust  this 
to  compensate  for  the  student's  progress, 

Q.  Is  the  evaluation  based  on  the  student's  achievement  of  the 
requirement  and  meeting  the  suspense? 

A.  The  evaluation  is  done  subjectively  by  the  phase  leader  and  the 
faculty  consultants.  There  are  various  evaluation  points  through¬ 
out  the  phase.  It  may  be,  "At  this  point  the  student  will  present 
a  briefing  or  a  staff  paper.  This  is  the  point  at  which  I  would 
ask  the  phase  leaders  and  faculty  consultants  to  be  there  to 
observe  the  presentation." 

Q.  The  Chaplains  School  is  using  groups  of  about  12  and  you  are 
using  groups  of  5.  Why  do  you  use  the  smaller  group? 

A.  It  is  harder  to  hide  in  a  five-man  group.  For  the  short  time 
he  stays  in  one  phase  he  might  be  able  to  sandbag  and  stay  out  of 
the  problem.  Also  in  most  staff  sections  in  which  the  faculty  has 
had  experience,  this  experience  shows  that  the  most  effective  work¬ 
ing  group  is  about  five.  With  five  people  every  man  is  sucked  in. 

He  can't  stay  out.  The  group  won't  let  him  because  if  one  man 
drops  out  that  is  a  20%  increase  in  workload.  A  point  of  interest 
is  that  we  have  two  allied  officers  in  the  class.  One  is  Australian 
and  handles  the  language  well.  The  other  is  a  Turkish  officer  who 
speaks  the  language  well  but  feels  that  he  has  trouble  writing. 

The  small  group  environment  is  perfect  for  him  because  he  is  not 
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the  least  bit  reluctant  to  try  his  less  than  perfect  English  since, 
if  the  group  does  not  understand  him,  they  can  ask  over.  He  is 
working  well  in  this  type  environment. 

Q.  Are  the  situations  which  you  assign  to  the  students  given  in 
writing  or  orally  and  in  what  form  are  they  given? 

A.  They  come  in  all  sorts  of  forms  such  as  a  letter,  DF,  buck  slip, 
or  written  note.  To  give  you  an  example,  last  year  during  Phase  V, 
the  Installation  Phase,  which  was  getting  near  the  end  of  the  course, 
the  class  was  given  an  assignment.  It  was  a  handwritten  note  saying, 
"I  am  considering  ordering  a  study  for  an  automated  information 
system  for  this  installation,  Fort  McClellan.  Give  me  a  briefing 
on  8  April  at  1000  hours  of  about  30  minutes  on  what  makes  this 
installation  tick--who  does  what  to  whom.  Signed,  Gen  Enemark, 

DA,  IG."  Gen  Enemark  was  scheduled  to  be  here  at  that  time.  As 
it  turned  out  he  could  not  come  so  he  sent  the  assistant  IG,  Gen 
Hainblin.  For  about  two  days  the  students  struggled  with  questions 
like,  "What  does  he  want?  What  is  he  asking  about?  What  kind  of 
briefing  does  he  want?"  They  finally  decided  what  the  problem  was 
and  then  they  went  out  to  all  the  activities  on  post  and  found  out 
how  the  post  works.  Incidentally,  the  cooperation  of  other  people 
is  essential  for  this  type  of  learning. 
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CMARC  SOFT  SKILL  TRAINING  CONFERENCE 
The  In-Baaket  Teat:  Ita  Development 
and  Application  to  Military  Training 

Charles  W«  Kroen,  Jr. 

USA  Signal  Center  and  School 
Ft  Monmouth ,  NJ 

The  construction  of  performance  examinations  for  officer  courses  pr^ 
sents  a  unique  challenge  to  the  test  developer.  He  is  faced  with  a 
qualitatively  different  problem  from  that  which  exists  in  developing 
examinations  for  enlisted  personnel*  Managerial  and  administrative 
skills  are  usually  being  taught  rather  than  specific  equipment 
related  skills  such  as  the  troubleshooting  and  repair  of  electronic 
equipment.  Assessing  the  attainment  of  training  objectives  in  this 
context  is  not  a  clear  cut  matter.  Both  administrative  skills  and 
administrative  knowledge  have  to  be  measured.  The  real  test  of  the 
effectiveness  of  officer  training  is  not  merely  knowledge  of  effec¬ 
tive  managerial  practices  and  procedures  but  rather  the  ability  to 
translate  this  knowledge  into  action.  In  other  words,  a  situational 
test  is  needed  to  assess  managerial  and  administrative  ability. 

This  test  for  maximum  effectiveness  should  be  a  realistic  simulation 
of  the  actual  job  situation  that  the  student  is  being  trained  for. 

He  must  be  exposed  to  the  kinds  of  problems  and  required  to  make  the 
kinds  of  decisions  that  he  will  have  to  make  on  the  job.  Only  in 
this  way  can  we  obtain  an  accurate  gauge  of  the  effectiveness  of 
officer  training  and  feel  certain  that  we  are  preparing  the  officer 
to  face  the  kinds  of  challenges  he  will  be  exposed  to  in  the  field. 

One  of  the  most  promising  training  evaluation  instruments  that  can  be 
used  to  assess  officer  training  is  the  in-basket  test  which  was 
originated  by  Dr.  Norman  Frederiksen  and  his  colleagues  at  the  Edu¬ 
cational  Testing  Service. 

The  in-basket  test  obtains  its  name  from  the  in-basket  or  tray  usual¬ 
ly  situated  on  a  manager's  desk  in  which  letters,  reports,  memos,  and 
other  papers  are  deposited  for  his  attention  and  action.  The  contents 
of  the  tray  serve  as  the  problems  for  the  in-basket  test.  The  person 
reviews  and  takes  action  on  these  problems.  Thus,  the  in-basket  test 
is  essentially  a  job  performance  ;:est  of  managerial  and  administra¬ 
tive  ability.  It  projects  the  person  engaging  in  this  exercise  into 
a  specific  role  which  he  is  to  perform  for  a  given  period  of  time. 
Usually,  he  is  given  background  information  for  his  role,  in  other 
words,  who  he  is  and  how  he  got  to  be  put  into  the  given  situation. 
Other  background  information  such  as  an  organization  chart,  job 
descriptions  of  subordinates,  and  important  upcoming  events  are  also 
provided  in  order  to  help  the  person  gain  a  proper  perspective.  The 
important  point  in  providing  the  person  with  background  information 
is  to  make  the  situation  appear  as  realistic  as  possible  and  to  provide 
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him  with  the  necessary  information  to  deal  effectively  with  in-basket 
problems.  The  participant,  after  he  is  given  the  background  informa¬ 
tion  and  placed  in  a  particular  role,  now  gives  his  attention  to  and 
takes  action  on  the  various  pieces  of  correspondence  which  comprise 
the  in-basket  exercise.  The  actions  taken  on  these  problems  require 
him  to  exhibit  knowledges  and  skills  acquired  in  his  past  experience. 

In  taking  action,  the  participant,  on  a  separate  piece  of  paper, 
states  what  kind  of  action  he  is  taking  and  why  he  is  taking  it. 

In  collecting  problems  for  the  in-basket  test,  every  effort  is  made 
to  secure  actual  material  from  the  job  that  is  to  be  simulated.  There 
are  important  benefits  to  be  derived  from  this  procedure  both  in  terms 
of  face  validity  and  content  validity.  Face  validity  is  achieved 
when  the  participants  feel  that  they  are  cast  in  a  realistic  job  situ¬ 
ation  and  confronted  with  the  kinds  of  problems  they  will  face  in  the 
actual  on-the-job  situation.  This,  in  turn,  enables  them  to  become 
ego  involved  in  the  exercise  and  motivated  to  do  their  best,  that  is 
to  exhibit  the  managerial  and  administrative  ability  they  possess. 
Content  validity,  in  the  context  of  a  training  situation,  is  satis¬ 
fied  when  actual  on-the-job  problems  are  selected  for  the  in-basket 
exercise  and  are,  in  turn,  related  to  specific  training  objectives. 
Thus,  the  in-basket  exercise  is  tied  in  a  closed  loop  process  to  the 
systems  engineering  design  of  courses,  for  both  test  content  and  train¬ 
ing  objectives  are  derived  from  a  detailed  analysis  of  the  job 
performance  requirements.  In  essence,  the  in-basket  test  is  a  job 
sample  test  incorporating  certain  knowledges  and  skills  given  to  an 
officer  in  training  which  he  now  applies  to  a  realistic  simulation  of 
an  actual  job  situation. 

Originally  the  in-basket  test  was  developed  at  Maxwell  Air  Force  Base, 
Alabama,  to  measure  the  effect  of  instruction  upon  job  proficiency 
(Findley,  et  al  1954) .  From  this  modest  beginning,  the  in-basket 
technique  has  been  used  both  as  a  selection  device  and  a  training  tool 
in  such  companies  as  Procter  and  Gamble,  American  Telephone  and  Tele¬ 
graph,  IBM^  and  the  Mew  York  Port  Authority.  It  has  been  used  in 
assessment  and  training  for  both  first  line  supervisors  and  middle 
management.  In-basket  tests  have  been  constructed  to  measure  ability 
in  a  wide  variety  of  occupations  such  as  production  foremen,  veterans' 
administration  executives,  educational  administrators,  executive 
secretaries,  and  police  lieutenants.  (Lopez,  1966) 

In  the  US  Army  Signal  Center  and  School,  we  used  the  in-basket  test 
to  measure  the  effectiveness  of  officer  training  in  our  Communications 
Center  Operations  Course.  The  in-basket  is  used  as  an  end-of-course 
examination  to  measure  the  effectiveness  of  training  throughout  the 
entire  course.  The  purpose  o*  the  test  is  to  see  if  an  officer  can 
take  the  material  he  has  learned  and  apply  it  in  a  practical  way  in 
order  to  solve  problems  and  make  decisions  in  a  realistic  simulation 
of  an  on-the-job  situation. 
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Several  key  problems  had  to  be  met  for  the  successful  developraeat  of 
an  in-basket  test  for  this  course.  These  problems  include:  select¬ 
ing  the  course  for  application  of  the  in-basket  test,  specifying  the 
situation  and  the  role  in  which  the  officer  is  placed,  determining 
*~he  amount  of  information  to  give  to  the  officer,  selecting  and  con¬ 
structing  the  in-basket  problems,  developing  the  scoring  system,  and 
making  sure  the  test  is  reliably  scored. 

The  Communications  Center  Operations  Course  was  selected  as  the  course 
in  which  to  develop  the  in-basket  test.  The  rationale  for  this 
selection  lies  in  the  fact  that  the  officer  is  being  trained  in  this 
course  for  either  of  two  closely  related  job  assignments  (assignment 
to  a  tactical  or  fixed  Communications  Center)  not  for  a  host  of 
slightly  related  jobs  each  of  which  possesses  its  own  unique  chal¬ 
lenges.  Thus,  the  construction  of  the  in-basket  problems  can  be  keyed 
to  a  specific  field  assignment. 

A  problem  still  existed,  however 5  of  whether  to  construct  two  in- 
basket  tests,  each  representing  a  specific  job  or  to  somehow  combine 
these  two  tests  into  one.  The  latter  alternative  was  selected  for  it 
was  felt  that  to  use  two  in-basket  tests  would  be  prohibitive  in  terms 
of  testing  time.  To  make  the  situation  as  realistic  as  possible  and 
to  present  the  widest  range  of  problems  possible,  it  was  decided  to 
create  the  role  position  of  the  Officer- in-Charge  (OIC)  of  a  tactical 
Coimunications  Center  that  was  in  transition  to  a  fixed  Communications 
Center.  Thtis,  he  was  involved  in  moving  from  field  vans  to  permanent 
buildings.  Further  realism  was  added  to  the  situation  by  providing 
the  officer  with  the  kind  of  information  he  would  have  a  real  life 
job  situation.  His  role  situation  was  described  together  with  the 
setting  he  was  in  and  the  equipment  he  possessed.  All  necessary 
forms,  regulations,  and  publications  were  provided.  In  addition,  he 
was  given  a  personnel  roster,  the  chain  of  command,  a  diagram  of  his 
present  facility,  and  a  diagram  of  his  future  facility.  All  this 
material  with  the  exception  of  classified  material  was  given  to  the 
officer  the  night  before  testing  to  insure  his  becoming  thoroughly 
familiar  with  the  role,  the  background  situation,  and  the  test 
requirements . 

Specific  instructions  given  to  the  officer  prior  to  taking  the  in- 
basket  test  require  him  to  be  in  his  place  of  work  and  to  take  action 
on  the  papers,  memos,  and  forms  in  his  in-basket.  Each  problem  has 
to  be  solved  and  all  necessary  action  has  to  be  recorded  on  proper 
forms  in  accordance  with  applicable  regulations.  If  he  wants  to 
delegate  certain  routine  actions  zo  subordinates,  he  has  to  designate 
which  subordinate,  why  that  subordinate,  and  give  him  detailed  in¬ 
structions  on  the  actions  to  be  accomplished.  Thus,  the  items  are 
structured  in  such  a  way  as  to  force  him  to  take  action.  He  is  plaped 
into  a  decision-making  situation  and  the  quality  of  his  decisions  can 
be  evaluated. 
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Aa  a  first  step  in  construction  of  the  in-basket  test,  instructors 
were  asked  to  select  the  most  critical  training  tasks  for  each  of 
the  six  phases  of  the  Communications  Center  Operations  Course*  These 
phases  are:  Logistics  and  Personnel,  Tactical  Communications  Centers, 
Tape  Relay,  Automatic  Data  Processing,  AUTODIN,  and  Fixed  Communica¬ 
tions  Centers.  The  in-basket  problems  will  reflect  these  training 
tasks.  Visits  were  made  to  Communications  Centers  both  at  Fort 
Monmouth  and  in  Philadelphia  to  observe  the  day-to-day  activities  of 
the  Center  and  to  collect  material  that  would  be  suitable  for  develop¬ 
ment  of  in-basket  problems. 

From  these  activities  and  from  the  experiences  of  the  teaching  per¬ 
sonnel,  five  problems  were  developed  for  the  in-basket  test.  The 
first  problem  deals  with  COMSEC  logistics  and  facility  approval  and 
has  the  officer  in  transition  from  a  tactical  to  a  fixed  station 
Communications  Center.  He  is  required  to  order  fixed  station  equip¬ 
ment,  documents,  set  up  required  security  measures,  and  obtain 
security  approval.  The  next  problem  is  an  inspection  problem  in 
which  the  officer  has  to  brief  his  Commanding  Officer  on  the  fre¬ 
quency  and  criteria  for  a  COMSEC  inspection  and  also  the  Command 
responsibility  with  regard  to  the  inspection.  The  third  problem 
deals  with  a  COMSEC  insecurities  problem  in  which  there  was 
unauthorized  techniques  used  in  encoding  a  message.  The  OIC  is 
required  to  determine  the  type  of  insecurity  and  affix  responsibil¬ 
ity,  make  a  proper  report  of  the  security  violation,  and  recommend 
corrective  action. 

In  the  fourth  problem  (COMSEC  accounting) ,  he  is  issued  an  actual 
box  of  CRYPTO  material  and  has  to  sign  for  it,  inventory  what  is  in 
the  box,  check  to  make  sure  all  the  documents  are  complete,  report 
on  any  missing  material,  and  destroy  duplicated  material. 

The  Standard  Operating  Procedures  (SOP)  problem  is  the  final  problem 
in  the  in-basket  test.  It  requires  the  OIC  to  construct  an  SOP  in 
outline  form  for  a  new  system  of  wide  band  terminals  to  be  installed 
in  the  Commanding  General's  office. 

In  all  of  these  problems,  the  officer  is  given  the  same  kind  of 
information  he  would  have  if  he  were  on  the  actual  job.  He  is  not 
told  what  course  of  action  to  take  and  then  graded  on  how  well  he 
carries  out  these  actions.  Rather,  he  has  to  analyze  the  problem, 
determine  the  correct  course  of  action  to  take,  and  describe  how  he 
would  carry  out  these  actions  to  the  desired  ends. 

Additional  realism  is  gained  when  these  problems  are  presented  on 
actual  Army  forms  such  as  routing  and  transmittal  slips,  disposition 
forms,  CGMSEC  material  report  forms,  cryptographic  message  forms, 
etc. 


III-17 


Vv 


The  officer  deals  with  each  problem  by  completing  the  required  forms 
which  he  obtains  from  a  table  on  which  all  the  forms  he  will  need  to 
complete  the  test  are  assembled  together  with  certain  distractor 
forms  which  have  no  useful  purpose  in  this  test.  He  also  has  a  supply 
f  paper  available  for  recording  decisions  and  carrying  out  actions 
that  are  not  readily  tied  to  particular  forms.  These  written 
responses  serve  as  a  basis  for  Judging  his  problem  solving  and 
decision  making  ability. 

Each  in-basket  problem  has  several  correct  decisions  to  be  made  and 
courses  of  action  to  be  taken.  It:  is  scored  on  a  nine-point  scale 
with  anchor  points  being  established  along  the  scale  at  points  1,  3, 

3 5  7,  and  9.  A  judgment  is  made  as  to  which  decisions  for  a  given 
problem  are  the  crucial  ones  and  these  are  given  more  weight  when  the 
solutions  to  the  problem  are  scored.  Thus  for  a  given  problem,  a 
person  would  receive  9  points  if  everything  was  correct,  7  points  if 
only  a  few  minor  decisions  and  actions  were  not  taken,  5  points  if 
all  critical  decisions  and  actions  were  taken  but  minor  decisions  and 
actions  were  not  taken,  3  points  if  some  of  the  critical  decisions 
and  actions  were  taken,  and  1  point  if  only  minimal  work  was  per¬ 
formed  on  the  problem. 

A  panel  of  judges  who  are  subject  matter  experts  specify  the  quality 
range  of  decisions  and  actions  for  each  problem.  These  are  assembled 
into  an  administrator's  manual  so  that  the  scorer,  in  evaluating  a 
participant's  solution  to  a  problem,  compares  his  response  to  the 
point  on  the  scale  to  which  it  more  closely  corresponds  and  assigns 
the  resulting  value  as  the  score  for  that  problem. 

Before  the  in-basket  test  can  be  put  into  operational  use,  scorer 
reliability  has  to  be  determined  and,  in  fact,  should  be  perfect, 
that  is,  three  or  four  scorers  should  score  a  participant's  in-basket 
test  independently  and  their  evaluations  should  agree.  If  they  do 
not  agree,  which  is  typically  the  case  when  the  in-basket  test  is 
first  scored,  discussion  should  be  held  among  the  scorers  to  discover 
the  points  of  difference  and  to  resolve  these  differences.  Once  the 
scoring  techniques  have  been  refined,  the  administrator's  manual 
should  be  updated  to  reflect  these  changes.  This  procedure  will 
allow  scorers,  who  use  the  administrator's  manual  as  a  guide,  to 
objectively  score  the  in-basket  test. 

Thus  far,  my  presentation  has  been  concerned  with  the  use  of  the 
in-basket  test  as  a  tool  to  evaluate  ability  to  be  trained  as  a 
Communications  Center  Officer.  I.  has  other  uses.  Parts  of  it  can 
be  given  as  a  diagnostic  tool  during  various  phases  of  a  course  so 
as  to  discover  student  deficiencies  in  mastering  course  content.  This 
information  could  lead  to  a  rewriting  of  the  Program  of  Instruction 
to  stress  certain  concepts  that  the  students  have  difficulty  grasping. 
Feedback  on  the  results  of  the  in-basket  test  can  be  given  to  trainees 


SO  that  they  can  gain  valuable  insights  into  their  managerial  and 
administrative  behavior.  Also,  they  can  get  an  idea  of  their  strong 
and  weak  points  in  the  areas  of  problem  analysis  and  decision  making. 

In  closing,  it  seems  to  me  that  the  in-basket  test,  by  providing  a 
realistic  sample  of  the  kinds  of  Job  skills  and  knowledge  expected 
of  trainees,  may  be  profitably  adopted  by  other  service  schools  who 
seek  a  more  meaningful  way  to  assess  the  expected  managerial  and 
administrative  behavior  of  their  graduates. 
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Questions  and  Discussion  Following  Dr,  Kroen's  Presentation 


Q.  When  the  student  response  is  scored  or  graded  does  che  student 
h:  /e  an  opportunity  for  discussion?  Is  there  any  negotiation  of 
grades? 

A.  There  is  about  an  hour's  discussion  after  the  in-basket  test. 

The  student  does  have  a  chance  to  reclama.  After  a  period  of  time 
the  solutions  are  pretty  well  established.  Initially,  modifications 
have  to  be  made  in  the  scoring  because  you  never  really  come  up  with 
all  the  contingencies  and  solutions  that  can  be  generated, 

Q.  How  did  you  establish  your  standards  for  grading? 

A.  Strictly  arbitrary.  The  five  out  of  nine  on  the  nine-point 
scale  translates  to  70  percent.  We  could  make  the  scale  any  value 
such  as  80  points . 

Q.  What  constitutes  satisfactory?  What  criteria  do  you  use? 

A.  For  example:  in  determining  a  security  leak  for  an  improperly 
sent  message,  there  are  four  or  five  decisions  that  an  officer  has 
to  make.  The  experts  will  generally  agree  on  what  the  critical 
decisions  are.  There  are  other  decisions  that  are  not  so  critical 
but  should  be  made.  You  can  get  into  a  corner  by  trying  to  make  too 
fine  a  gradation.  What  we  try  to  do  is  have  a  dichotomy.  One  group 
of  decisions  are  those  which  are  absolutely  critical  and  important. 
Other  groups  of  decisions  are  extensions  of  the  critical  decisions; 
in  other  words,  things  which  would  have  to  be  carried  out  to  have  an 
absolutely  successful  problem.  So  what  you  do  is  to  determine  which 
decisions  are  absolutely  critical  and  which  are  needed  but  not 
absolutely  critical.  For  example,  if  there  are  five  decisions  to 
be  made  and  three  are  critical,  these  three  would  equate  to  five  on 
the  nine-point  scale  and  the  remaining  minor  decisions  would  be 
used  to  determine  the  extra  points  from  five  to  nine  on  the  scale. 

If  he  misses  one  of  the  critical  decisions  he  would  get  less  than 
five . 

Q.  What  happens  in  real  life  when  you  make  a  mistake  on  something 
like  that  is  that  you  end  up  with  a  bunch  of  nonconcurrences.  In 
effect  you  could  go  to  your  subject  matter  experts  and  ask,  "Do  you 
concur  or  nonconcur?"  If  you  get  a  bunch  of  nonconcurrences,  the 
thing  can  end  up  back  in  the  student's  lap.  That  is  what  happens 
in  real  life  and  the  student  can  then  resolve  that  problem.  It 
would  seem  that  that  would  be  an  effective  soft  sell  of  the  scoring 
sys tern. 
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A,  Yes.  That  is  true.  One  of  the  things  ve  do  is  to  require  the 
student  to  pick  up  a  package  of  classified  material.  He  has  to 
inventory  the  material  and  see  \^at  is  there.  If  anything  is  miss¬ 
ing  there  are  certain  type  reports  he  has  to  make.  If  there  are 
duplications  there  are  certain  type  reports  required.  If  he  is 
going  to  sign  for  a  package  he  has  to  make  sure  that  the  package 
has  not  been  tampered  with.  We  have  packages  all  made  up  and  the 
student  is  brought  into  the  room  to  sign  for  the  packages.  He  is 
observed  to  see  if  he  checks  for  tampering  and  so  forth.  If  he  just 
signs  without  checking  that  is  not  good  enough. 

Q.  You  use  the  in-basket  as  a  test.  What  method  of  teaching  do 
you  use  to  lead  up  to  and  get  the  student  ready  for  this  type  of 
test? 

A.  The  way  we  usually  teach  our  officer  courses  is  to  present  the 
material  in  lectures  and  conferences  and  then  test  him,  usually, 
with  multiple  choice  type  questions.  You  are  not  really  challeng¬ 
ing  him  with  any  kind  of  problems  or  actual  job  situations.  Then 
he  gets  out  in  the  real  world  and  faces  the  real  world  situations 
through  the  type  of  problems  we  are  talking  about  here.  What  we 
want  to  do  is  check  his  ability  to  solve  these  real  world  problems. 

Q,  In  your  in-basket  test  do  you  try  to  test  the  officer’s  pro¬ 
gramming  or  process  skill?  Are  you  concerned  with  the  method  with 
which  he  solves  the  problems? 

A.  Not  really.  You  are  talking  about  the  method  with  which  a 
student  arrives  at  a  certain  decision.  We  are  not  concerned  with 
the  method  he  uses.  We  are  concerned  with  whether  or  not  he  does 
it  at  all.  I  am  not  sure  in  my  own  mind  (in  spite  of  some  of  the 
speakers  yesterday)  that  we  are  at  a  point  in  our  technology  that 
we  can  teach  process.  We  have  to  teach  content  first.  After  this 
is  learned  then  maybe  we  can  turn  our  attention  to  teaching  procesk. 

Q.  Do  you  use  the  test  as  a  diagnostic  and  prescription  device? 

A.  We  have  not  really  explored  its  use  as  a  diagnostic  tool 
although  1  think  it  could  be  used  for  that.  Of  course  the  wrong 
responses  can  be  identified  but  to  find  out  what  led  the  student 
astray  would  require  some  interaction  between  the  student,  the 
instructors,  and  possibly  other  students.  It  could  be  that  instruc¬ 
tion  was  vague  or  that  the  student  Is  doing  things  wrong  or  not 
enough  time  was  allowed  for  certain  areas.  These  types  of  things 
have  to  be  looked  into. 

Q.  Do  you  have  immediate  feedback  with  the  in-basket  test? 


A.  Yes.  Nhen  1  talk  about  feedback  1  seen  that  lonedlately  after 
the  officer  takes  his  test  there  is  a  critique  session  where  they 
go  over  the  problens  and  all  the  decisions  and  the  why  of  the 
courses  of  action  and  %diat  is  right  and  idtst  is  wrong.  The  students 
.'eel  that  the  exam  is  not  Mickey  Mouse  and  that  the  exam  does  bear 
out  what  they  should  do.  It  correlates  with  idiat  officers  do  in 
other  kinds  of  situations.  They  feel  that  it  is  a  worthiAile  and 
realistic  test. 
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CON/OtC  SOFT  SKILL  TRAINING  CONFERENCES 
Learning  Centers 

Clarence  C«  Newsom 
Education  Advisor 

United  States  Army  Aviation  School 
Fort  Rucker,  Alabama 

Major  Stanfield  of  the  Infantry  School  at  last  year's  training 
conference  gave  a  very  fine  report  on  that  school's  Individual 
Learning  Center.  He  discussed  their  experience  with  selection 
of  hardware,  development  of  software  and  operation  of  the  facility. 
Atteiqptlng  to  avoid  undue  duplication,  I  will  tell  you  about  our 
experience  with  the  Learning  Center  at  the  Aviation  School.  I'd 
like  to  emphasize  that  we  do  not  claim  to  have  the  ultimate  answer, 
but  we  are  pleased  with  the  results  of  the  facility.  I  will  also 
tell  you  a  little  about  the  learning  centers  at  Forts  Wolters  and 
Bennlng. 

The  use  of  learning  centers  at  service  schools  first  came  to  our 
attention  lAen,  in  January  of  1969,  the  Primary  Helicopter  School 
at  Fort  Wolters  opened  one  for  Vietnamese  students.  That  venture 
was  such  a  success  that  one  was  opened  for  American  students  In 
January  of  1970.  By  this  time,  we  were  seriously  Interested  In 
determining  If  such  a  facility  would  be  of  value  to  the  Aviation 
School.  We  began  a  study  of  our  situation  and  visited  the  center 
at  Fort  Wolters  and  those  at  some  Air  Force  and  Navy  Installations. 

At  Fort  Rucker,  we  teach  a  wide  variety  of  courses  ranging  from 
Warrant  Officer  career  courses  and  Aviation  Medical  Officer  courses 
through  Initial  entry  and  graduate  flight  courses  to  enlisted 
specialists  and  mechanics  courses.  The  learning  environment  ranges 
from  modern  multi-media  classrooms,  renovated  World  War  II  buildings, 
open  hardstands,  radar  and  flight  simulators  to  airborne  cockpits 
spread  over  an  area  of  about  60  by  40  miles. 

A  basic  decision  was  made  to  establish  a  learning  center  for  the 
purpose  of  providing  remedial  and  supplemental  Instruction  of  an 
Individual  basis,  supporting  specific  courses.  Initially,  the  main 
effort  was  to  support  the  Instrument  phase  of  the  Initial  entry 
flying  cotirse.  Support  to  other  courses  was  planned  to  be  phased 
on  a  priority  basis.  The  decision  to  support  instrument  training 
as  first  priority  was  based  on: 

1.  An  analysis  of  flight  and  academic  performance  data  which 
Indicated  this  was  a  problem  area  In  what  we  consider  to  be  our  "bread 
and  butter"  fll^t  course. 


2.  The  tasks  being  learned  vere  largely  proeedwal  In  nature 
and  lent  themselves  to  individualization. 

3.  A  number  of  prograemd  texts,  training  aids,  and  devices  were 
e  -ailable  to  support  the  training. 

4.  Students  in  the  course  were  indoctrinated  in  using  the  learning 
center  at  Fort  Wolters. 

Personnel  to  operate  the  center  and  to  develop  individual  lessons  case 
from  "our  hide."  Staffing  is  still  a  major  problem  in  that  we  recognize 
a  need  for  24  people  and  Just  this  year  we  were  authorized  six  on  the 
TEA.  The  staff  is  divided  into  a  Lesson  Development  Section  and  a 
Training  Section. 

The  learning  center  was,  and  still  is,  housed  in  an  air  conditioned 
recreation  hall  type  building  convenient  to  student  living  quarters. 

It  was  made  as  attractive  and  confortable  as  possible  throu^  the 
"self-help"  program-— materials  and  paint  furnished  by  the  engineers 
and  labor  furnished  by  the  staff. 

We  were  caotlous  in  buying  equipment  because  of  lack  of  experience  and 
the  firm  belief  that  hardware  was  a  little  value  without  effective 
learning  programs.  I  will  not  discuss  selection  of  hardware  because 
I  believe  you  should  select  what  is  needed  for  your  particular  situa¬ 
tion.  Let  it  be  known  that  you  are  interested  and  you  will  have  a 
proliferation  of  brochures,  catalogs,  and  salesmen. 

The  USAAVNS  Learning  Center  opened  in  January,  1971,  equipped  with: 

1.  Study  carrels. 

2.  Slide  and  movie  projectors. 

3.  Tape  recorders. 

4.  ETV  on  call. 

5 .  Aircraft  cockpits . 

The  staff  attempts  to  provide  a  choice  of  individualized  slefpaced 
lessons  presented  by: 


1.  TV. 

2.  Sound/slide  programs. 

3.  Tape  cassettes. 

4.  Super  Sum  film. 


111-24 


c 


5.  Programed  texts. 

6.  Live  Instrxictors. 

From  a  modest  coverage  of  Instrument  subjects,  we  now  have  lessons 
available  with  a  choice  of  media  in  the  following  areas: 

1.  Instruments. 

2.  Aircraft  systems. 

3.  Tactics. 

4.  Contact  flight. 

5.  Air  traffic  control. 

6.  Enrichment  lessons. 

Operating  hours  of  the  learning  center  are  0730-2200,  Monday  through 
Thursday;  0730-1630,  Friday;  and  1200-2200,  Sunday.  These  hours  were 
chosen  as  a  result  of  a  survey  of  students  attending  the  facility. 

The  study  revealed  that  evening  is  the  most  popular  time  with  peak 
attendance  at  1930. 

Since  opening,  emphasis  has  been  on  voluntary  attendance,  but  there 
is  provision  for  referral  by  Instructors.  A  slip  with  the  problem 
identified  is  given  the  student.  He  presents  it  at  the  learning 
center,  appropriate  notation  is  made  on  it  and  he  returns  the  slip 
to  his  Instructor.  About  15  percent  of  those  attending  are  referred. 

The  remainder,  of  course,  are  volunteers.  The  average  visitor  spends 
about  one  and  a  half  hours  in  the  center.  To  encourage  attendance,  it 
was  necessary  to  publicize.  This  was,  and  still  is,  done  through 
briefings,  flyers,  daily  bulletin,  post  newspaper,  local  radio  and 
TV  programs  and  any  other  means  possible.  After  two  years,  people 
still  "discover”  the  learning  center! 

It  is  apparent  from  the  foregoing  discussion  that  student  attendance 
records  are  maintained.  In  addition,  lesson  usage  records  are  kept. 
Students  are  also  asked  to  complete  critique  forms  indicating  needs, 
preferences,  and  Interests.  These  records  assist  in  making  decisions 
regarding  operating  hours,  lessons  to  stock,  methods  and  media. 

Though  not  it's  primary  mission,  the  learning  center  serves  as  a 
laboratory  for  innovation.  A  branch  facility  was  established  at  one 
of  the  base  airfields  to  support  the  synthetic  flight  trainer  syllabus 
in  basic  Instrument  training.  Students  waiting  on  the  ground  while 
their  stick  buddies  were  flying  or  in  synthetic  trainers  were  adhiinistered 
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programed  texts  designed  to  prepare  them  for  their  next  synthetic  trainer 
flight.  This  served  to  increase  student  retention,  saved  instructor 
briefing  time,  and  capitalised  on  practical  application  in  Che  trainer. 
When  success  of  this  operation  was  established,  it  was  transferred  to 
"he  flight  department  and  became  a  pcnoanent  part  of  the  course.  A 
cockpit  iBock-19  that  was  animated  to  show  realistic  Instrument  readings 
was  duplicated  by  a  commercial  firm  and  two  of  the  devices  are  now 
used  as  procedural  trainers  in  a  course  of  instruction.  El^erience 
gained  in  preparing  individualized  programs  proved  to  be  perticulerly 
valuable  in  converting  three  omchanics  courses  to  the  self*paced 
configuration.  There  have  been  ocher  exaag>les  and  we  hope  there,  will 
be  more  where  improvements  learned  in  the  learning  center  can  be  sioved 
to  the  classroom. 

Thus  far,  I  have  not  addressed  cost  and  funding.  The  staff  requisitioned 
and  drew  equipment  where  it  could  be  found;  they  fabricated,  improvised, 
and  borrowed.  The  connand  made  funds  available  within  the  budget. 

From  support  people  we  heard,  "You  can't  do  it,  it's  not  on  the  TDk." 

And  from  training  people  we  ^ard,  "It's  a  flash  in  the  pan  and  won't 
work."  We  did  get  people  and  materials  to  operate  the  learning  center 
and  it  has  worked  well.  About  sixty  users  visit  the  learning  center 
on  an  average  day.  Approximately  15  percent  of  the  users  are  staff  and 
faculty  seeking  refersher  training.  Coonents  from  staff  and  faculty 
as  well  as  students  continue  to  be  laudatory. 

The  education  advisors  at  the  Primary  Helicopter  School  and  the  Infantry 
School  have  furnished  some  recent  information  about  the  learning  centers 
at  Forts  Wolters  and  Banning.  Both  are  engaged  in  innovations  that  are 
interesting. 

Fort  Wolters  is  employing  gaming/siinulators  to  capture  the  student's 
Interest  and  to  stimulate  his  learning.  The  learning  center  is  used 
as  one  of  several  specialised  classrooms.  Other  rooms,  or  modules,  are 
fitted  to  support  a  particular  subject  such  as  navigation  or  map  reading. 

Fort  Banning  expanded  it's  learning  center  from  30  to  50  carrels  some 
time  ago  and  this  month  they  are  noving  a  131  carrel  center  on  the 
second  floor  of  Infantry  Hall.  In  addition  to  the  usual  equipment, 
each  carrel  will  have  a  student  response  system.  They  plan  to  expand 
the Review  and  enrichment  mission  to  Include  use  of  the  faculty  for 
core  curriculum  programs. 

To  those  who  are  considering  establishing  a  learning  center,  I  have 
some  suggestions.  Begin  with  assignment  of  a  mission  to  meet  your 
own  particular  requirement.  Do  not  duplicate  our  facility  nor  that 
of  anyone  else.  Visit,  look  and  study,  then  select  equipment  and 
programs  to  support  your  special  situation. 

Wt‘  w(>lcoim*  your  visit  to  Fort  Kuckor  and  aru  most  happy  to  show  you 
what  we  have  and  what  we  are  doing. 


III-26 


SLIDES 


1  -  Aircraft,  flag  and  sign 

2  -  Vietnamese  Student 

3  -  Fort  Rucker  Gate 

4  -  USAAVNS  with  wings 

5  -  School  Mission 

6  -  Initial  Entry  Rotary  Wing  Student  Fort  Wolters  ^  Fort  Rucker 

7  -  Local  Area 

8  -  1st  Airfield 

9  -  2d  Airfield 

10  -  New  Classrooms 

11  -  U-21  and  CH-54 

12  -  T-41 

13  -  Maintenance  Training 

14  -  Synthetic  Flight  Training  System 

15  -  Cockpit 

16  -  Radar  Simulator 

17  -  Learning  Center  Mission 

18  -  Building 

19  -  Equipment 

20  >  Methods 

21  -  Lessons 

22  -  Interior  View 

23  -  Interior  View 

24  -  TV  -  Multiple  Listening 

25  -  MK  IV 

26  -  Sound  Slide 

27  -  Procedure  Trainer  with  MK  IV 

28  -  Fort  Wolters  Mailbox 

29  -  Fort  Wolters  Building 

30  — -  Fort  Wolters  Table  with  Programed  Texts 

31  >  Fort  Wolters  Student  at  Carrel 

32  -  Fort  Wolters  Inside  View 

33  -  Fort  Wolters  Helicopter 

34  -  Fort  Wolters  Mock-up  of  Cockpits 

35  -  Fort  Wolters  Sight  Picture  Device 

36  -  Fort  Wolters  Flight  Control  Device 

37  -  Fort  Wolters  Game/Slmulation  Title 

38  -  Fort  Wolters  TH-55  Approach  Game 

39  -  Fort  Wolters  Electrical  System  Preflight  Game 

40  -  Fort  Wolters  Student  Playing  Game 


Questions  end  Discussion  following  Mr.  Newsom's  Presentation 


Q.  We  have  had  the  problem  of  needing  something  new  and  getting 
wtiat  we  need  to  match  up  with  what  we  already  have.  How  do  you 
solve  that  problem? 

A.  That  is  one  of  the  problems  you  have;  making  sure  that  your 
equipment  is  compatible.  The  Director  of  Coonunications-Electronics 
will  do  his  best  to  keep  you  honest  in  this  area,  because  each  time 
you  talk  about  a  new  piece  of  equipment  coming  into  the  inventory, 
they  are  going  to  shudder.  It  complicates  their  maintenance  prob¬ 
lem.  Our  experience  has  been  good  as  far  as  maintenance  is  concerned. 
We  have  had  very  little  trouble.  We  have  had  more  trouble  with  the 
equipment  we  bought  and  found  we  had  no  use  for.  For  example,  we 
bought  a  Super  8mn  camera  with  sound  to  make  our  own  snvies.  It 
sounds  good  and  looks  good--but  steer  clear. 
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CONARC  SOFT  SKILL  TRAINING  CONFERENCE 
Computer  Slmulatione 


Major  Bernard  Meacham 
Chief,  Automation  Branch, 
Supply  Career  Department 
US  Army  Quartermaster  School 
Fort  Lee,  Virginia 


The  purpose  of  this  presentation  is  to  introduce  you  to  computer 
assisted  instruction  as  it  is  conducted  at  the  Quartermaster  School, 

Fort  Lee,  Virginia,  But,  before  I  go  into  descriptions  of  the 
various  simulations,  let  me  give  you  a  little  history. 

Planning  for  computer  assisted  instruction  at  the  QM  School  began  in 
May  1966  with  a  feasibility  study  to  determine  areas  of  instruction 
that  would  benefit  the  greatest  from  the  use  of  the  computer.  Con¬ 
sideration  was  given  to  the  expanding  employment  of  automatic  data 
processing  equipment  in  the  Army's  supply  system  and  the  need  to 
employ  instructional  methods  that  relate  to  this  automated  environ¬ 
ment.  After  many  months  of  study  and  preparation  the  School's  Data 
Systems  Division  was  established  and  given  the  missions  of  coordinat¬ 
ing  the  educational  use  of  automatic  data  processing  equipment  and 
programming.  In  July  1969  an  intra-service  support  agreement  with 
the  Army  Logistics  Management  Center  for  joint  use  of  their  computer 
mainframe  was  affected.  (Slide  #2)  The  Center's  UNIVAC  Series  70 
computer  is  dedicated  to  educational  activities.  Using  appropriate 
elements  of  the  feasibility  study  as  guidelines,  the  School  selected 
remote  terminal  equipment  and  proceeded  with  their  prototype  simula¬ 
tions.  The  first  simulation  was  placed  on  the  air  in  May  1971. 

Currently,  the  School  has  two  fully  tested  simulations;  COSINES  and 
SIMIASS  (Slide  #3)  . 

COSINES  (Slide  #4)  is  designed  for  the  enlisted  student  who  has  little 
or  no  experience  in  supply  and  emphasizes  receipt,  issue,  reporting, 
and  accounting  procedures  detailed  in  AR  725-50.  The  simulation 
supports  the  Stock  Control  and  Accounting  Specialist  Course  (MOS 
76P20)  •  Personnel  ^o  graduate  from  this  course  are  qualified  for 
assignment  to  Materiel  Management  Centers  as  exception  editors.  As 
such,  they  will  be  required  to  use  computer  generated  management 
information  to  correct  erroneous  input  and  in  an  emergency  operate 
punch  card  machinery. 

During  the  course,  students  receive  a  series  of  instructional  blocks 
and  comprehensive  practical  exercises,  and  are  exposed  to  a  variety 
of  instructional  methods,  including  programmed  text  and  audio/visual 
(Slide  #5  and  5a).  At  the  conclusion  of  each  block  a  computer-supported 
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exercise  (COSINES)  is  given  to  further  Illustrate  and  reinforce  teach¬ 
ing  points.  Each  exercise  is  composed  of  a  series  of  problems 
graduated  in  complexity  and  geared  so  that  each  student  maintains  his 
own  dialogue  with  the  computer,  through  the  use  of  80  column  punch 
<  irds.  Each  student's  problems  are  his  own  and  hgve  absolutely  no 
effect  on  fellow  students. 

In  COSINES,  the  computer  is  programmed  to  produce  to  each  student 
punch  card  documents  that  contain  errors.  Students  must  make 
appropriate  corrections  and  return  the  documents  to  the  system.  If 
proper  corrections  were  made,  a  new  document,  containing  different 
errors,  will  be  produced.  If  the  input  document  was  not  completely 
corrected,  the  student  will  receive  an  error  notice  and  must  make  the 
required  corrections  before  he  receives  the  next  problem. 

An  important  feature  of  CC^INES  is  the  presence  of  remote  input /out¬ 
put  terminals  In  the  classroom  (Slide  #6) .  The  simulation  can  be 
conducted  simultaneously  in. four  separate  locations.  Terminals  are 
connected  to  the  computer  mainframe  by  "half  duplex"  C2  rated  voice 
grade  lines.  The  printers  are  connected  to  the  computer  by  "full 
duplex"  C2  rated  voice  grade  lines. 

This  photograph  (Slide  #7)  shows  a  typical  classroom,  note  the  remote 
equipment  at  the  front  of  the  classroom.  This  is  a  (Slide  #8)  close- 
up  of  one  of  the  IBM  1050  input/output  terminal  systems;  on  your  left 
is  an  IBM  1057/1058  printing  card  punch.  This  unit  provides  output 
for  the  simulation  and  can  be  shifted  to  independent  keypunching 
mode.  The  unit  on  your  left  and  to  the  front  is  an  IBM  1056  card 
reader.  It  feeds  and  reads  punched  cards  and  provides  input  capa¬ 
bility.  This  unit,  (Slide  #8a)  a  UNIVAC  740  printer,  is  used  to 
print  management  reports  and  can  print  up  to  300  lines  per  minute 
depending  on  the  number  of  characters  per  line. 

(Slide  #9)  Simulations  such  as  COSINES  increase  the  number  of  trans¬ 
actions  available  to  students  and  provide  rapid  responses  to  students 
in  the  event  they  input  erroneous  information  (Slide  #10) « 

(Slide  #11)  These  are  COSINES  objectives:  (Slide  #12)  SDflASS  is 
directed  to  senior  enlisted  and  junior  officer  students.  It  simu¬ 
lates  management  activities  required  of  a  commodity  manager  in 
support  of  an  independent  corps  under  coiid>at  conditions.  Student^ 
received  approximately  40  hours  of  classroom  instruction  in  inven¬ 
tory  management  before  entering  the  exercise.  Each  manager  is 
uniquely  identified  by  a  manager  number  and  assigned  his  own 
separate  group  of  items.  From  the  time  the  student  "signs-on"  to 
the  computer  from  his  assigned  remote  terminal,  until  the  time  the 
simulation  is  ended,  16  hours  later,  situations  are  presented  by  a 
time-delay  technique  from  the  computer  held  scenario. 
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Students  are  confronted  with  decision  making  problems  covering 
declaration  of  excesses,  generation  of  supply  directives,  prepare- 
tions  and  submission  of  requisitions  and  follow-up  documents  and 
establishment  and  adjustment  of  requisitioning  objectives. 

SIMIASS  equipment  is  aligned  as  shown  here  (Slide  #13) .  Each  blue 
block  represents  a  student  work  area.  Those  areas  with  two  termi¬ 
nals  can  support  five  two-man  teams  or  10  students.  Three  teams 
or  six  students  are  usually  assigned  to  those  work  areas  that  have 
one  terminal. 

The  area  in  red  denotes  the  instructor  control  center.  The  terminal 
shown  is  for  instructor  use  only.  On  the  other  hand,  the  printer 
supports  both  the  instructor  and  the  student.  Information  requested 
by  the  instructor  or  the  student  that,  \dien  printed,  will  be  in 
excess  of  six  lines  will  be  provided  by  the  printer.  Otherwise,  all 
input  and  output  is  provided  by  the  terminals .  The  area  in  yellow 
shows  the  tie  with  the  computer  main-frame  (SIMTASS  Title  Slide  #14) . 

The  exercise  has  the  following  features  that  make  it  a  good  instruc¬ 
tional  vehicle: 

a.  A  student  can  advance  at  his  own  pace  without  affecting  the 
progress  of  other  students. 

b.  With  Instructor  assistance,  a  student  can  be  advanced  to  any 
situation  in  the  exercise  and  be  taken  back  to  any  unplayed  situation. 

c.  The  program  provides  the  instructor  with  a  full  range  of 
Information  to  assist  him  in  controlling  the  exercise. 

(1)  The  instructor  terminal  is  notified  when  a  student  has 
input  problems,  such  as  format,  or  has  selected  the  incorrect  decision 
(these  notices  are  made  on  the  student's  third  try). 

(2)  The  computer  is  programmed  to  keep  track  of  student  progress. 
By  Inquiring,  the  iiwtructor  can  determine  the  status  of  one  or  all 
students.  He  will  be  provided  the  location  within  the  exercise,  the 
time  that  the  current  situation  was  started  and  the  total  number  of 
errors  made. 

(3)  In  certain  Instances  the  computer  will  automatically  provide 
a  status  report  to  the  Instructor  when  the  student  fails  to  follow  a 
preferred  processing  sequence. 

These  are  photographs  of  the  SIMIASS  facility: 


«  • 
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a.  (Slide  #15)  The  Control  Center.  Left  to  right  is  the  IBM 
data  conmunication  terminal  and  the  UNIVAC  740  printer*  Also  in 
this  area  is  a  reference  library  and  a  distribution  center. 

b.  (Slide  #16)  Student  Work  Area.  Note  the  two  IBM  1052  termi¬ 
nals  at  the  front  of  the  area.  ^ 

c.  (Slide  #17)  Close^un  of  a  Terminal.  All  data  communication 
terminals  have  standard  typewriter  keyboards.  It  takes  approximately 
five  minutes  to  prepare  a  student  for  operation  of  his  assigned 
terminal. 

SIMTASS  objectives  are  shown  here:  (Slide  #18) 

(Slide  #19)  The  QH  School  has  these  three  simulations  under  develop¬ 
ment:  All  three  simulations  will  be  conducted  in  the  same  facility 
previously  discussed  for  SIKTASS. 

SIMWIM  will  (Slide  #20)  simulate  a  hypothetical  inventory  control 
point  dealing  in  wholesale  supply  management,  and  is  an  adaptation 
of  CAL06SIM,  an  ADIC  simulation.  The  exercise  will  employ  the  con¬ 
cept  of  one  manager  having  the  responsibility  for  many  items.  Major 
and  secondary  items,  and  repair  parts  have  been  chosen  so  that  the 
student  is  exposed  to  items  of  varying  degrees  of  importance. 

Each  student  will  be  provided  a  smarting  position  on  each  of  his 
items,  and  his  sole  responsibility  is  to  establish  or  maintain  a 
satisfactory  supply  position.  To  do  this,  he  must  project  and  realize 
the  long-term  effects  of  his  decisions  since  SIMWIM  is  designed  to 
cover  a  4-year  cycle  graduated  by  time  frames.  During  the  simulated 
4-year  period  the  student  will  be  exposed  to  a  variety  of  problems  that 
will  effect  the  supply  position  of  his  items.  Increased  demands, 
natural  disasters,  strikes,  and  a  war  are  but  a  few  examples.  The 
computer  will  automatically  alert  the  student  to  changes  in  the  status 
of  his  items.  He  must  anticipate  and  respond  to  these  changes  by 
appropriately  applying  the  more  than  50  management  decisions  available 
to  him. 

Desirable  features  of  the  simulation  are: 

a.  The  exercise  will  be  designed  to  accommodate  individual  or 
team  play. 

b.  Students  must  live  with  all  decisions  made,  but  can  improve 
his  supply  position  through  intensive  management  techniques. 

c.  Each  student  will  receive  an  ef fectivenesb  score. 

d.  Students  can  work  at  their  own  pace  without  effecting  the 
progress  of  other  students. 
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SIMWIM  programming  is  approximately  907.  complete.  We  project  that 
system  testing  will  begin  In  January  1973. 


(Slide  #21)  Simulation,  petroleum  laboratory  evaluation  exercise 
(SIMPLEX)  will  be  a  simulation  of  petroleum  product  acceptability 
where  the  student  Is  confronted  with  decisions  as  to  whether  a 
product  is  worthy  of  its  Intended  use.  By  using  the  computer  to 
supply  laboratory  analysis,  the  student  can  concentrate  on  how  to 
use  the  results  of  the  analysis. 

As  In  the  other  simulations  discussed,  the  Instructor  will  be 
alerted  to  student  difficulties  and  during  the  graded  portion  the 
computer  will  automatically  monitor  and  grade  each  student.  The 
objective  of  SIMPLEX  Is  to  Increase  the  number  of  decisions  each 
student  must  make  about  product  acceptability,  thus  better  preparing 
him  for  th<>  field  petroleum  laboratory.  Our  Data  Systems  Division 
turned  thlb  exercise  over  to  the  Petroleum  Department  for  Instructor 
testing  and  OJT  on  6  Nov  72.  Formal  airing  of  the  exercise  Is 
scheduled  for  Feb  1973. 

(Slide  #22)  Simulation,  Spectrometrlc  Analysis  Exercise  (SIMSAX) 
will  be  a  petroleum  simulation  Involving  spectrometrlc  oil  product 
analysis.  This  Involves  analyzing  oil  samples  from  operating  equip¬ 
ment,  and  advising  operating  activities  of  maintenance  procedures 
Indicated  by  the  analysis.  Throughout  the  exercise  students  will 
be  required  to  consider;  kinds  of  metal  wear,  threshold  limits,  and 
trend  analysis.  A  portion  of  the  simulation  will  be  graded. 

SIMSAX  Is  currently  suspended  awatlng  further  Information  from  the 
field  due  to  DOD  policy  changes  affecting  the  4  branches  of  the 
military  service. 

(Slide  #23)  The  installation  of  SIMWIM,  the  Inventory  management 
exercise,  SIMPLEX,  and  SIMSAX,  the  petroleum  exercises,  will  Increase 
the  number  of  simulations  In  the  QM  School  to  five. 

Looking  Into  the  future,  our  Instructional  departments  have  deter¬ 
mined  that  these  areas  (Slide  #24)  should  be  considered  for  some  type 
of  computer  assisted  Instruction  (Slide  #25). 

The  simulations  that  I  have  discussed  have  enhanced  the  quality  of 
Instruction  at  the  QM  School.  The  most  Ideal  learning  combination, 
obviously  would  be  one  teacher  -  one  student.  Then  the  student 
receives  Individual  attention  and  Immediate  teacher  response  when 
needed.  This  we  cannot  accomplish  due  to  a  lack  of  Instructors.  We 
believe  that  we  can  effectively  bridge  most  of  this  gap  through  the 
use  of  computer  assisted  Instruction. 
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LIST  OF  SLIDES 


1.  QM  School  Crest 
*2.  Photo  of  AIMC  Computer 

3.  Explanation  of  COSINES  and  SIMIASS 

4.  COSINES  Title 

*5.  Photo  of  audio /visual  equipment  (COSINES) 

6.  COSINES  Equipment  Alignment 
*7.  Photo  COSINES  Classroom 
*8.  Photo  COSINES  Input/Output  Terminals 
*8a.  Photo  COSINES  Printer 
9.  COSINES  Title  (see  4  above) 

*10.  Photo  of  Student  (COSINES) 

11.  COSINES  Objectives 

12.  sntlASS  Title 

13.  SIKTASS  Equipment  Alignment 

14.  SIMIASS  Title  (see  11  above) 

*15.  Photo  SIMIASS  Control  Center 
*16.  Photo  SIMTASS  Work  Area 
*17.  Photo  SIMTASS  Terminal 

18.  SIMIASS  Objectives 

19.  List  of  Developmental  Systems 

20.  SIMWIM  Title 

21.  SIMPLEX  Title 

22.  SIMSAX  Title 
*23.  QM  School  Crest 

24.  CAI  Areas  Under  Consideration 
*25.  QM  School  Crest 

*The8e  are  photographs,  an  explanation  of  each  is  provided  below.  An 
example  of  the  other  slides  listed  are  attached  hereto. 

SLIDE  NUMBER  EXPLANATION 


2.  Photo  of  a  portion  of  the  AI2fC  computer  -  Univac  Series 

70. 

5.  Shot  of  an  Audio /Visual  work  area  showing  the  35  mm 

projector  and  the  screen. 

7 .  View  from  the  rear  of  a  COSINES  classroom  showing 
students  at  work  and  terminal  equipment. 

8.  Close-up  of  an  IBM  1057/1058  Printing  Card  Punch, 

IBM  1056  Card  Reader. 

8a.  Close-up  of  the  Univac  70  printer. 


SUDE  NUMBER 


EXPLAMATIOW 


10.  A  Student  showing  disappointment, 

15.  View  of  the  IBM  1052  Keyboard  Printer,  and  the 
Unlvac  70  Printer. 

16.  View  of  a  SIMIASS  Work  Area  showing  students  at 
work  and  remote  terminals. 

17.  Close-up  of  terminal  keyboard. 
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Slide 


Slide  18 


Slide  19 


Slide  24 


Questions  snd  Discussion  Folloving  Major  Mescham's  Presentation 


Q.  You  mentioned  six  terminals  and  then  you  said  the  simulations  are 
individually  paced.  Does  that  mean  that  you  can  get  six  displays 
off  of  one  terminal? 

A.  No.  We  have  eleven  terminals.  In  each  work  area  there  are  ten 
Individuals  and  two  terminals.  There  are  five  work  groups  in  each 
work  area.  We  work  in  teams  for  cosnunications  purposes  but  each 
team  stands  alone. 

Q.  Does  each  student  or  team  have  a  code  number? 

A.  Yes.  Each  team  has  a  code  number  that  is  identified  to  the  cosf 
puter.  He  signs  on  for  example  as  Team  A  1.  In  any  given  work 
area  no  more  than  one  team  will  be  working  on  the  sam  problem. 

Q.  Do  you  have  any  projection  of  the  comparable  cost  of  totally 
utilizing  this  system  or  are  you  doing  some  things  with  this  system 
which  you  could  not  do  wxth  other  systems? 

A.  We  are  doing  things  we  could  not  do  without  the  computer. 

One  of  the  siain  objectives  to  allow  the  student  to  operate  as 
close  to  the  real  environment  as  possible.  We  don't  duplicate  what 
the  computer  does  by  letting  the  student  do  it.  In  depots,  in 
supply,  in  supply  management,  the  computer  does  most  of  the  work. 

The  sttident  works  wn  the  exceptions. 
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CONARC  SOFT  SKILL  TRAINING  C(»IFERENC£ 
Results  of  the  SMNART  Army  Training  Survey 

Harold  L.  Moon 
HuaRRO  Division  Ko«  2 
Fort  Knox,  Kentucky 


CONARC  has  been  interested  for  a  long  time  in  developing  a  reliable 
procedure  for  selecting  the  most  satisfactory  training  methods  and 
media,  and  HuoRRO  Division  2  at  Fort  Knox  began  a  Work  Unit  called 
MEDIA  two  years  ago  in  1970.  Later,  in  1971,  Work  Unit  COST  was 
also  begun*  COST  was  to  develop  training  cost-analysis  procedures. 

An  exploratory  study  in  MEDIA  attempted  to  identify  an  empirical 
basis  for  a  methods-media  selection  manxial.  This  was  done  by  a 
survey  of  the  literature.  The  results  showed  that  neither  suitable 
guidelines  nor  sufficient  en^irical  data  exist  which  can  be  used  for 
a  reliable  methods-media  selection  procedure. 

Therefore,  in  December  of  1971  --  after  a  HumRRO  -  COHARC  conference 
on  work  units  MEDIA  and  COST  --  the  two  units  were  combined  into  an 
expanded  program,  now  called  Work  Unit  SMMART,  which  stands  for 
Developing  Criteria  for  the  Selection  of  Meth^s  and  Media  by  Army 
Trainers. 


The  work  on  training  cost  is  proceeding  well,  and  that's  all  to  be 
said  about  it  today,  because  we  are  concerned  in  this  conference 
with  methods  and  media.  So  remaining  remarks  will  be  about  what  we 
have  done  in  Work  Unit  SMMART  toward  development  of  a  methods-media 
selection  procedure  and  what  remains  to  be  done. 

Last  spring  we  prepared  a  Draft  Technical  Report  on  The  State  of 
Knowledge  pertaining  to  The  Selection  of  Cost-Effective  Methods 
and  Media,  which  is  now  being  edited  for  publication.  We  also  pre¬ 
pared  a  research  plan  for  determining  an  empirical  basis  for  the 
selection  of  media,  which  would  require  several  years  to  complete. 
Meantime,  if  we  didn't  do  something  else,  the  Army  would  be  without 
anything  from  HumRRO  to  help  in  selecting  methods  and  media.  So  as 
part  of  the  research  plan,  we  proposed  to  develop  in  a  relatively 
short  time  a  selection  procedure  based  on  the  experience  of  quali¬ 
fied  Army  training  personnel. 

To  do  that,  we  followed  the  lead  of  R.W.  Walker  of  the  MARTIN- 
DENVER  COMPANY  on  an  approach  he  reported  in  the  Human  Factors 
Journal.  The  steps  in  Walker's  approach  are  on  the  first  page  of 
the  handout  materials.  Walker  first  asked  his  training  staff  to 
list  criteria  they  used  in  selecting  methods  and  media.  Then  he 
asked  them  to  list  the  methods  anc  media  they  use  or  thought  might 
be  useful  in  training.  Finally,  to  determine  the  value  of  the 


methods  and  media,  he  asked  his  training  personnel  to  rate  each 
method  and  medium  in  relation  to  each  of  the  selection  criteria  they 
listed.  The  ratings  were  done  on  a  five^point  scale,  with  5  being 
the  highest  value  and  1  the  lowest.  Then  Walker  averaged  the  rat¬ 
ings  of  each  method  and  medium  for  each  criterion.  This  gave  him  a 
consensus  value  for  each  method  and  medium,  based  on  his  trainers' 
judgments. 

Although  Walker's  project  was  rataer  skimpy,  we  decided  that  his 
approach  has  good  potential  if  thoroughly  and  carefully  developed. 

This  is  what  we  proposed  to  do  for  the  ^rmy,  with  the  cooperation  of 
qualified  Amy  training  personnel. 

As  the  first  step  in  applying  this  approach,  we  prepared  a  training 
survey  questionnaire. 

The  questionnaire  had  several  purposes.  One  was  to  identify  Army 
training  personnel  by  name,  school,  job  title,  job  function  or  func¬ 
tions,  and  type  of  experience  with  Army  training.  Another  purpose 
was  to  get  a  list  of  criteria  used  in  the  Army  for  selecting  methods 
and  media,  and  a  third  was  to  learn  which  methods  and  media  Army  train¬ 
ing  personnel  use,  have  used  or  would  use  if  they  had  the  opportunity. 

CONARC  sent  out  a  total  of  483  questionnaires  --  21  of  each  of  23 
CONARC  schools.  Of  these,  443  were  completed  and  returned  from  21 
schools.  None  were  received  from  the  Chaplain  School  and  the  Infantry 
School  because  they  apparently  did  not  receive  the  questionnaire.  To 
those  of  you  who  participated  in  this  survey,  we  hearlly  thank  you 
for  your  splendid  cooperation. 

Of  the  443  completed  questionnaires  received,  15  were  not  usable 
because  they  were  from  persons  involved  only  in  basic,  or  entry, 
training  --  and  we  are  concerned  in  this  project  with  only  the  three 
levels  of  courses  which  follow  entry  training.  In  these  levels,  we 
are  also  interested  in  both  hard-  and  soft-skill  courses.  These 
three  levels  and  two  types  of  skills  make  up  the  six  Army  course 
types  as  defined  in  Regulation  350-100-1  --  the  regulation  on  develop¬ 
ment  of  training  systems. 

The  six  Army  course  types  are  Level  I  hard-  and  soft-skill  courses  for 
students  who  have  had  entry  training;  Level  II  hard-  and  soft-skill 
courses  for  students )who  have  had  Advanced  Individual  (AIT);  and 
Level  III  hard-  and  soft-skill  courses  for  students  who  have  had  unit 
experience . 

The  questionnaires  represented  423  persons  whom  we  were  able  to 
classify  into  four  major  categories.  These  categories  were  instruc¬ 
tors,  supervisors,  systems  engineers,  and  educational  advisors. 


We  were  also  able  to  classify  the  428  persons  according  to  the  three 
course  levels  and  the  hard  and  soft  skills  «dilch  divided  the 
Individuals  Into  the  following  12  categories. 

28  were  Involved  with  hard  skills  on  course  Level  I; 

9  with  hard  skills  on  Level  II; 

11  with  hard  skills  on  Level  III; 

39  were  Involved  with  hard  skills  on  all  three  course  levels. 

12  worked  with  soft  skills  on  Level  1; 

3  with  soft  skills  on  Level  II; 

117  with  soft  skills  on  bevel  III;  and 
91  with  soft  skills  on  all  three  course  levels. 

12  worked  with  both  hard  and  soft  skills  on  Level  I; 

2  with  hard  and  soft  skills  on  Level  II; 

13  with  hard  and  soft  skills  on  Level  III;  and 

91  with  both  hard  and  soft  skills  on  all  three  course  levels. 

We  further  analyzed  the  Information  received  In  the  questionnaires 
for  purposes  which  will  be  stated  as  we  move  along. 

First  we  noted  that  a  total  of  29  criteria  for  the  selection  of 
methods  and  siedla  were  listed  by  respondents.  These  are  shown  In 
Table  1  (see  page  III-55) • 

On  the  right  of  the  table  are  the  percentages  of  the  428  Individuals 
vdio  listed  each  criterion,  starting  with  55  percent  at  the  top  down 
to  one  at  the  bottom  of  the  column.  These  percentages  were  used  for 
ranking  the  criteria.  The  ranks  are  in  the  column  of  figures  on  the 
left  side  of  the  table.  One  at  the  top  represents  the  criterion  most 
frequently  mentioned,  and  28.5  at  the  bottom  represents  the  two 
criteria  least  frequently  mentioned.  Since  they  had  the  same 
frequency,  they  were  tied  for  lowest  rank.  You  will  note  that  there 
are  other  tied  ranks  in  the  column. 

Both  the  percentages  and  ranks  represent  all  the  428  respondents. 
ks  you  mig^t  expect,  however,  there  were  some  differences  in  the 
percentages,  and  therefore  in  the  rankings  of  the  criteria,  between 
the  various  groups  of  Army  training  personnel  represented  by  the 
questionnaires.  Because  of  these  differences,  it  was  important  for 
us  to  know  how  well  the  groups  agreed  on  the  criteria. 

Table  2  shows  on  the  left  the  groups  we  compared  to  see  how  well 
they  agreed  on  the  rankings.  Note  first  that  these  groups  were 
classified  according  to  skills,  course  levels,  and  main  job  func* 
tlons  with  which  the  respondents  were  Involved. 
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TABLE  2 


SELECTION  CRITERIA  -  GROUP  RANKING  COMPARISONS 


GROUPS 

COTIFFI 

CIENT 

X2 

SIGNIF . 
28  d.f. 

SKILLS 

HARD  -  SOFT  -  BOTH  i 

1 

1  .937 

1 

78.71 

>>  .001 

COURSE  LEVELS 

! 

I  &  II  -  III  I 

i 

.913 

1 

1  - 

76.69 

>>  .001 

MAIN  JOB  FUNCTION  i 

[ 

INSTRUCTORS  -  ! 
SUPERVIS.  -  SYSTEMS  ; 
ENG.  j 

! 

.947 

79.55 

>>  .001 

In  the  skills  category,  we  compared  three  groups:  those  involved  with 
hard  skills  only;  those  involved  with  soft  skills  only;  and  those 
involved  with  both  hard  and  soft  skills. 

In  the  course- level  category,  we  compared  two  groups:  those  involved 
with  courses  on  Levels  I  and  II;  and  those  involved  with  Level  III 
courses.  Levels  I  and  II  were  combined,  because  there  were  only  five 
persons  in  Level  II. 

In  the  Main-Job-Function  category,  we  compared  Instructors,  Super- 
visots,  and  Systems  Engineers*  In  this  case,  Educational  Advisors 
were  included  in  the  supervisor  group. 

We  used  the  Kendall  Concordance  Test  to  see  how  well  the  groups 
agreed  on  their  rankings  of  the  methods  and  media  selection  criteria. 
This  test  yields  a  rank  correlation  coefficient. 

As  you  can  see,  the  correlation  coefficients  for  the  groups  in  the 
three  categories  are  .937,  .913,  and  .947  --  all  of  vdiich  are  very 
high. 

To  test  the  significance  of  the  correlation  coefficients,  we  calcu¬ 
lated  Chi  Square  values,  which  were  also  very  high;  all  were 
significant  at  levels  far  better  than  .001  level  of  confidence. 

Another  way  to  interpret  these  results  is  to  conclude  that  the  groups 
compared  came  from  a  homogenous  population  --  or  from  the  same 
population. 


So  much  for  the  methods  and  media  selection  criteria  as  obtained  by 
the  questionnaires. 


Now  about  the  methods  and  media  information.  The  questionnaire  con¬ 
tained  a  list  of  102  methods  and  media,  and  the  respondents  were 
asked  to  check  those  they  use,  have  used  or  would  use  in  Army  train¬ 
ing. 

The  information  was  quite  useful  for  several  purposes.  First,  we  were 
able  to  identify  the  frequencies  at  which  various  groups  checked  the 
items.  This  is  important,  because  we  assume  that  higher  frequencies 
indicate  those  groups  which  probably  have  higher  degrees  of  experi¬ 
ence  in  training.  We  assume  also  that  the  higher  frequencies  indicate 
a  greater  awareness  of  methods  and  media,  and  we  hope  that,  if  our 
assumptions  are  correct,  the  higher  degrees  of  experience,  and  aware¬ 
ness  of  methods  and  media,  also  indicate  higher  degrees  of  training 
expertise. 

To  determine  meaningful  frequencies  for  comparing  groups,  we  first 
calculated  mean  percent  frequencies  for  six  groups,  as  shown  in 
Table  3. 


TABLE  3 

METHODS -MEDIA  PERCENT  FREQUENCIES 


MEAN 


COURSE 

NO.  OF 

PERCENT 

STAND. 

SKILLS 

LEVELS 

RESPOND. 

FREQ. 

I»V. 

1. 

HARD 

DIFF. 

48 

51 

25 

2. 

HARD 

ALL 

39 

51 

27 

3. 

SOFT 

DIFF. 

132 

58 

22 

4. 

SOFT 

ALL 

91 

60 

22 

5.  HARD 

-  SOFT 

DIFF. 

27 

51 

24 

6.  HARD 

-  SOFT 

ALL 

91 

65 

20 

TOTAL 

428 

58 

22 
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The  first  group  represents  48  respondents  who  were  involved  with  hard 
skills  in  the  different  course  levels.  That  is,  we  put  all  those  who 
work  with  hard  skills  separately  in  Level  I,  Level  II,  and  Level  III 
in  this  Group  1* 

Group  2  id  made  up  of  39  individuals  who  work  with  hard  skills  in  all 
three  course  levels. 

Group  3  are  the  132  persons  who  work  with  soft  skills  separately  in 
the  three  different  course  levels. 

Group  4  are  the  91  persons  involved  with  soft  skills  in  all  three 
course  levels. 

Group  5  are  27  persons  who  work  with  both  hard  and  soft  skills  but  in 
the  different  course  levels;  and 

Group  6  are  the  91  persons  who  work  with  both  hard  and  soft  skills  in 
all  three  course  levels. 

The  mean  percent  frequencies  were  derived  by  adding  up  all  the  times 
each  member  of  a  group  checked  each  of  the  102  methods  and  media  in 
the  list  you  have  in  the  handout  materials,  and  then  by  dividing  the 
sum  by  102,  the  number  of  items. 

The  results  show  the  average  percentage  of  the  number  of  times  each 
of  the  102  items  was  checked  by  the  members  of  each  group. 

The  bottom  line  in  Table  3  shows  the  mean  percent  frequency  for  all 
428  respondents,  which  is  58. 

Now  note  that  only  two  groups  **  4  and  6  have  a  mean  percent 
quency  higher  than  the  overall  mean  percent  frequency  of  38,  Three 
are  below  the  overall  mean,  and  one  is  the  same  as  the  overall  mean. 

We  are  interested  in  the  two  groups  which  have  a  mean  percent  fre¬ 
quency  above  the  overall  mean  —  Groups  4  and  6. 

Therefore,  as  stated  earlier,  we  assume  that  Group  6  contains  at 
least  some  individuals  who  have  a  higher  degree  of  training  experi¬ 
ence  than  other  persons  in  the  overall  population  of  428.  This 
assumption  is  also  reasonable,  especially  for  Group  6,  for  members 
of  that  group  work  with  both  hard  and  soft  skills  in  courses  on  all 
three  levels  of  training. 

As  with  the  methods  and  media  selection  criteria,  we  had  to  find  how 
well  these  groups  agreed  on  their  relative  frequencies  of  checking 
the  methods  and  media  items.  Without  doing  this,  we  would  have  no 
basis  for  confidently  proceeding  further. 
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TAl^LE  4 


METHOD -MEDIA  GROUP  RANKING  COMPARISONS 


100  d«f. 

COURSE  RHO  t-Value 

SKILLS  LEVELS  MO.  COEFF.  t-Value  .01  Level 


1.  HARD 

2.  HARD 

DIFF. 

ALL 

48 

39 

.933 

25.833 

2.626 

3.  SOFT 

4.  SOFT 

DIFF. 

ALL 

.888 

19.287 

2.626 

5.  HARD  &  SOFT 

6.  HARD  &  SOFT 

DIFF. 

ALL 

27 

91 

.796 

13.131 

2.626 

12  OTHER  COMBINATIONS 

....  .  _  , 

.666 

CO 

.939 

8.823 

50 

27.061 

2.626 

Table  4  shows  the  same  groups  as  in  Table  3,  but  the  added  data  are 
rank  correlation  coefficients  for  comparing  the  groups  and  t-test 
values  as  checks  on  the  significancies  of  the  correlations. 

We  checked  not  only  the  correlations  between  the  pairs  of  the  six 
groups  but  for  the  12  other  possible  combinations  of  those  groups. 

As  you  can  see,  the  coefficients  range  ^rom  .666  to  .939.  The 
t-values  also  are  highly  significant.  Cx.  t-value  of  only  2.626  is 
required  for  significance  at  the  .01  level  of  confidence,  and  the 
t-values  obtained  are  significant  at  least  at  the  .001  level  of 
confidence. 

So,  as  with  the  selection  criteria,  we  concluded  with  high  confi¬ 
dence  that  all  the  groups  are  from  the  same  population;  all  are  in 
high  agreement. 

Now  we  must  get  back  to  Groups  4  and  6  in  Table  4  which  we  identified 
as  probably  containing  persons  who  have  the  highest  degrees  of  train¬ 
ing  experience. 

It  is  important  to  identify  these  persons,  at  least  by  major  job 
function,  because  we  need  their  cc operation  as  Army  training  experts 
in  the  next  step  of  our  research.  As  Walker  did  with  his  training 
staff,  we  need  them  to  rate  the  methods  and  media  in  relation  to 
the  methods  and  media  selection  criteria. 
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In  our  search  for  the  training  experts,  we  must  turn  to  the  four 
groups  we  identified  according  to  major  job  function  the  Educa¬ 
tional  Advisors,  Systems  Engineers,  Supervisors,  and  Instructors. 


TABLE  5 


GROUPS  FOR  COMPARING  METHCH) -MEDIA  RANKS 


JOB  FUNCTICmS 

NO.  FROM 
GROUP  6 

NO.  FROM 
GROUP  4 

TOTAL 

NO. 

NO.  LEFT 
IN 

GROUP  6 

ED.  ADV1S(XIS 

MSM 

5 

17 

79 

SYSTEMS  ENG. 

HIH 

4 

17 

78 

SUPERVlSCmS 

14 

3 

17 

77 

INSTRUCTORS 

8 

9 

17 

83 

These  groups  are  shown  in  Table  5.  The  total  of  Educational  Advisors 
was  only  17.  We  found  12  of  them  in  our  highly  prized  Group  6,  and 
5  in  the  next  best  group  -  Group  4. 


To  make  fair  comparisons  between  che  advisors,  engineers,  super¬ 
visors  and  Instructors,  we  selected  17  representative  members  of  all 
groups  because  there  were  only  17  Educational  Advisors.  We  got  all 
the  persons  in  each  group  that  were  in  Group  6,  and  selected  the 
remaining  number  at  random  from  Group  4. 

Notice  that  in  the  column  on  the  far  right  are  the  number  left  in 
Group  6  after  selecting  out  the  members  of  the  occupational  groups. 


TABLE  6 

METHODS -MEDIA  GROUP  PERCENT  FREQUENCIES 

MEAN 

PERCENT  STAND. 


ED.  ADVISCHIS 

■a 

■Ml 

12 

GROUP  6 

mm 

HH 

22 

SYSTEMS  ENG. 

mm 

HBI 

16 

GROUP  6 

H9 

— 

22 

SUPERVISORS 

17 

aai 

26 

GROUP  6 

77 

wssm 

19 

INSTRUCTORS 

■a 

49 

24 

GROUP  6 

Kal 

64 

19 

ALL  RESPONDENTS 

428 

58 

22 
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Table  6  shows  the  mean  percent  frequencies  of  all  the  groups  under 
consideration. 


Mote  that  Educational  Advisors  have  the  highest  mean  percent  fre¬ 
quency  of  84.  Systems  Engineers  have  the  next  hipest  mean  percent 
frequency  of  77. 

If  our  assumption  is  correct  that  higher  mean  percent  frequencies 
of  checking  methods  and  media  indicate  higher  degrees  of  training 
experience,  and  if  our  hope  is  not  in  vain  that  they  also  Indicate 
greater  training  expertise  --  then  clearly,  Educational  Advisors 
probably  are  the  top  experts  we  seek,  with  Systems  Engineers  close 
runners  up. 

To  find  out  the  lowest  number  of  Educational  Advisors  or  Systems 
Engineers  we  can  use  as  experts  for  rating  methods  and  media  in  rela¬ 
tion  to  methods  and  media  selection  criteria,  we  first  selected  at 
random  two  groups  of  Educational  Advisors  and  two  groups  of  Systems 
Engineers  with  five  in  each  group  and  cbmpared  them  by  calculating 
rank  correlation  coefficients. 


TABLE  7 

COMPARISON  OP  ED,  ADVISORS  AMD  SYSTEMS  EM6. 
GROUPS  OF  FIVE  BACH 


100  d.f. 

RHO  t-Value 

GROUPS _ MO,  COEFF.  t-Value  .01  Level 


ED.  ADVISORS  (1) 

ED.  ADVIS(HIS  (2) 

5 

5 

1 

.590 

7.301 

2.626  ■ 

SYSTEMS  ENG.  (1) 
SYSTEMS  ENG.  (2) 

5 

5 

.397 

4.330 

2.626 

marm 

4.500 

FOUR  OTHER 

■h 

to 

2.626 

COMBINATIONS 

■1 

6.322 

Table  7  shows  the  results  «  All  the  coefficients  ranged  from  .397  to 
.590.  The  .397  coefficient  may  seem  low,  but  we  can  see  by  its 
t-value  that  the  significance  is  better  than  the  .01  level  of 
confidence. 

To  push  the  test  of  possibility  to  che  lower  limit,  we  selected  at 
random  two  groups  of  Educational  i4dvisors  and  two  groups  of  Systems 
Engineers  with  only  two  persons  in  each  group. 


tabu:  8 


COMPARISON  OF  ED.  ADVISORS  AND  SYSTEMS  ENG. 
GROUPS  OF  TWO  EACH 


100  d.f. 

RHO  t>Vaiue 

GROUPS _ NO.  C0EF7.  t-Value  .01  Level 


ED.  ADVISORS 
ED.  ADVISORS 

(1) 

(2) 

2 

2 

.199 

2.026 

2.626 

SYSTEMS  ENG. 
SYSTEMS  ENG. 

(1) 

(2) 

2 

2 

.032 

0.318 

2.626 

Table  8  shows  the  results.  The  coefficients  are  very  low  and  neither 
Is  significant  at  the  .01  level  of  confidence,  vdtich  we  feel  Is 
necessary  If  we  are  to  get  reliable  ratings  of  the  methods  and  media. 

We  decided  to  make  one  more  comparison,  this  time  with  three  persons 
In  each  group. 


tabu;  9 

COMPARISON  OF  ED.  ADVIS(»IS  AND  SYSTEMS  ENG. 
GROUPS  OF  THREE  EACH 


100  d.f. 

RHO  t -Value 

CTOITPS _ NO.  COEFF.  t -Value  .01  Level 


ED.  ADVISORS  (1) 
ED.  ADVISORS  (2) 

3 

3 

.552 

6.626 

2.626 

SYSTEMS  ENG.  (1) 
SYSTEMS  ENG.  (2) 

3 

3 

.361 

3.871 

2.626 

ED.  ADVISmS  (1) 

-  SYSTEMS  ENG.  (2) 

3 

3 

.082 

0.825 

2.626 

THREE  OTHER 
COMBINATIONS 

.358 

to 

.468 

3.839 

to 

5.316 

2.626 

As  shown  In  Table  9,  all  the  correlation  coefficients  were  signifi¬ 
cant  at  the  .01  level  except  one  --  the  .082  coefficient  for  Educational 
Advisors  and  Systems  Engineers. 

So  we  decided  to  be  safe  and  not  settle  for  less  than  groups  of  five 
Educational  Advisors  who  have  consistently  shown  up  In  our  data  as 
Che  best  single  group. 
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This  does  not  mean  that  we  need  only  five  Educational  Advisors.  It 
Mans  that  three  or  four  groups  of  five  men  can  share  the  work,  If 
they  are  willing,  which  will  loake  the  burden  relatively  easy  to 
bear. 

Involved  In  the  next  step  are  the  ranking  of  selection  criteria  and 
the  rating  of  Mthods  and  media  to  be  done  by  Education  Advisors. 


SMIART  HANDOUT  MATERIALS 


R.  H.  Walker's  Approach* 


I.  He  first  asked  his  training  staff  to 
list  the  criteria  they  used  in 
selecting  methods  and  media. 

II.  Then  he  asked  them  to  list  the  methods 
and  media  they  used  or  thought  might 
be  useful  in  training. 

III.  Then  he  asked  his  staff  to  rate  each 
method  and  medium  in  relation  to 
each  of  the  selection  criteria  they 
selected  (on  a  five-point  scale) . 

IV.  Then  Walker  averaged  the  ratings  for 
each  method  and  medium,  to  get  a 
consensus  value  for  each,  based  on 
his  trainers'  Judgments. 


(*  Walker,  R.  W.  "An  Evaluation  of  Training  Methods  and 
their  Characteristics,"  Human  Factors.  1965,  2>  PP  347-354. 


NUHBER  OP  qUESnOIOlAIIB  BESPOMDeNTS 
IN  SKILL  AMD  COURSE— LEVEL  CATEGORIES 

28  -  Hard  Skllla,  Course  Level  I 

9  -  Hard  Skills,  Course  Level  II 

11  -  Hard  Skills,  Course  Level  III 
39  •  Hard  Skills ,  all  Course  Levels 

12  -  Soft  Skills,  Course  Level  I 
3  -  Soft  Skills,  Course  Level  II 

117  -  Soft  Skills,  Course  Level  III 
91  -  Soft  Skills,  all  Course  Levels 

12  -  Hard  &  Soft  Skills,  Course  Level  I 

2  -  Hard  &  Soft  Skills,  Course  Level  II 

13  -  Hard  &  Soft  Skills,  Course  Level  III 

91  -  Hard  &  Soft  Skills,  All  Course  Levels 

Total:  428 
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TABLE  I 


Percentage 

Criteria  Of  Population 


1 

Realism 

55 

2 

Experience  Level  of  Student 

42 

3 

Complexity  of  Subject  Matter  -  Task 

34 

4 

Ease  of  Administration 

31 

5.5 

Equipment  Availability 

30 

5.5 

Time  ' 

30 

7 

Training  Aid  Availability 

28 

8 

Effectiveness 

27 

9 

Cost 

25 

10 

Facilities  -  Transporting  Training  Aid 

24 

11 

Student  Interest 

23 

12 

Ease  of  Instruction  -  Simplicity 

18 

13 

Class  Size 

17 

14 

Task  Criticality 

15 

15 

Student  Participation 

14 

17 

Cost-Effective 

12 

17 

Relevance 

12 

17 

Nuiidl>er  of  Supporting  Personnel 

12 

19.5 

Durability  -  Length  of  Servlep 

11 

19.5 

Instructor  Qualifications 

11 

21 

Number  of  Teaching  Points  to  Which 

M/M  Apply  -  Adaptability 

9 

22 

Sense  Channel 

7 

23.5 

Class  Frequency 

4 

23.5 

Safety 

4 

26 

Ease  of  Evaluation 

3 

26 

Platform  Manhours 

3 

26 

Instructor  Preference 

3 

28.5 

Cost  of  Development 

1 

28.5 

Student-Instructor  Ratio 

1 
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PARI  III 

METB(»S  AKD  MEDIA  LIST 


Directions 

Please  consider  the  methods  end  media  listed  below,  and  check 
them  If  you  use  them,  have  used  them,  or  would  use  them  for  Army 
training. 

Methoda  and  Media 


(Check) 


1. 

Actual  equipment 

2. 

_ 

Advance  sheets 

3. 

Animated  cartoona 

4. 

Bar  charts 

5. 

Block  diagrams 

6. 

Cartoons 

7. 

Case  studies 

8. 

Cassette  TV 

9. 

Chalkboard 

10. 

Charts 

11. 

Closed-circuit  TV 

CAI: 

Computer  Assisted 
Instruction 

12. 

CAI:  general 

13. 

CAI:  drill  and  practice 

14. 

CAI:  practice 

15. 

CAI:  problem  review 

16. 

— 

CAI:  diagnosis  and 
prescription 

17. 

CAI:  tutorial 

18. 

CAI:  gaming 

19. 

CAI:  slmulatli-' 

20. 

CAI:  fact-finding 

21. 

CAI:  computation 

22. 

— 

CAI:  logical  problem 
solving 

23. 

CAI:  exploring 

(Check) 


24. 

Cimference 

25. 

Correspondence  study 

26. 

Cross  sections 

27. 

Denons  t  r  a  t  ions 

28. 

_ 

Diagrams 

29. 

_ 

Discussion  sessions 

30. 

Exhibits  and  displays 

31. 

_ 

Expert  panels 

32. 

Exploded  views 

33. 

_ 

Film,  San,  sound 

34. 

Film,  Smcti,  silent 

35. 

_ 

Film,  single  concept 

36. 

Film,  Ibomi 

37. 

Filmstrip,  silent 

38. 

_ 

Filmstrip,  sound 

39. 

Flannel  board 

40. 

Flow  charts 

41. 

Graphics 

• 

CM 

_ 

Guest  speakers 

43. 

Hand-out  sheets 

44. 

In-basket  simulation 

45. 

Instructor 

46. 

— 

Job  experience  training 
(hands  on  equipment) 

47. 

Laboratory  work 
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Methods  and  Media  (Continued) 


(Check) 

Language  Laboratory: 


48. 

Language  Lab:  general 

49. 

— 

Language  Lab:  audio 
active 

50. 

— 

Language  Lab:  audio 
pasalve 

51. 

— 

Language  Lab:  audlo- 
active-compare 

52. 

Lectures 

53. 

— 

Line  drawings  and  views 

54. 

Line  graphs 

55. 

Lists  of  key  words 

56. 

Magnetic  board 

57. 

Manuals 

58. 

Maps  (two  dimensional) 

59. 

_ 

Microfilm 

60. 

_ 

Microprojectors 

61. 

Microteaching 

62. 

Models 

63. 

On- the -Job  Training 

64. 

Opaque  projection 

65. 

Organization  charts 

66. 

Overhead  projector 

67. 

_ 

Photographs 

68. 

Pictures  (printed) 

69. 

Pie  charts 

70. 

vwwaM* 

Posters 

71. 

Practical  exercises 

72. 

Peer  Instruction 

73. 

Printed  materials 

PI: 

Programed  Instruction 

74. 

PI:  branched  type 

75. 

_ 

PI:  linear  type 

(Check) 


76. 

Radio 

77. 

Realistic  drawings 

78. 

Record  player 

79. 

Relief  maps 

80. 

— 

Reference  books 

81. 

Role  playing 

82. 

Sand  table 

83. 

Sdiematics 

84. 

- — 

Self  confrontation 

85. 

_ 

Seminar 

86. 

- 

Skeletal  views 

87. 

Sketches 

88. 

-  '  " 

Simulators 

89. 

Slides 

90. 

Sound-on-Slide 

91. 

Student  response  units 

92. 

Sumnary  sheets 

93. 

Tachistoscope 

94. 

Tape  recorder 

95. 

Teaching  machines 

96. 

Television 

97. 

Terrain  boards 

98. 

- 

Textbooks 

99. 

Training  devices 

100. 

Tutorial  method 

101. 

_ 

Venetian  blind 

102. 

—  ■■■  > 

Videotape  (recording 
and  playback 
capabilities) 
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Summary  of  Dlacuaaiot:  of  Mr.  Moon's  Presentation 


Much  discussion  was  generated  during  Mr.  Moon's  presentation  per* 
cicularly  concerning  the  structure  of  the  questionnaire  which  HunRRO 
had  sent  to  the  service  schools  to  obtain  information  and  data  for 
Work  Unit  SItfART.  Most  of  the  conferees  were  in  general  agreement 
that  the  questionnaire  asking  idiet  method  and  media  the  respondents 
had  used,  were  using,  or  would  uga.  was  too  open-ended  and  subject 
to  speculation  and  subjectivity  rather  than  an  objective  answer. 
Since  no  conditions  were  established  for  the  use  or  possible  use  of 
the  method  or  media  many  of  the  respondents  assumed  that  if  condi¬ 
tions  warranted  it^  the  method  or  media  would  be  used.  It  was  felt 
that  this  lack  of  specificity  probably  inflated  the  frequencies  of 
selection  for  the  various  methods  and  media.  It  was  pointed  out 
that  the  questionnaire  would  probably  have  served  its  purposes 
better  if  the  questions  had  related  only  to  the  past  and  present  and 
omitted  the  future  since  many  of  the  respondents »  in  projecting  into 
the  future,  speculated  that  conditions  in  that  future  may  be  such 
that  the  use  of  the  method  or  media  would  be  warranted. 
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MAJ  Richard  F.  Malone 


MAJ  Malone  is  currently  Chief  of  the  Training  Quality  Control  Branch,  Cur- 
ricxilum  and  Evaluation  Division,  Office  of  the  Director  of  Instruction,  US 
Amy  Adjutant  General  School.  A  graduate  of  Texas  Tech  Unniversity  (B.A. 
Speech  1963,  M.Ed,  1971) •  MAJ  Malone  has  served  in  a  vau*iety  of  assign¬ 
ments  rauiging  from  Battalion  Adjutant  in  Europe  to  Systems  Engineering 
Officer  at  USAAGS.  MAJ  Malone  has  attended  the  Automatic  Data  Processing 
Systems  Analysis  Course  and  the  AG  Officer  Advanced  Course  and  is  a  member 
of  Phi  Kappa  Phi  and  the  Acadeny  of  Management. 


Dr.  Jacobs  received  the  Ph.  D.  degree  from  the  university  of  Plttsburj^ 
in  1956.  Since  Joining  the  Human  Resources  Research  Organization,  he  has 
specialized  in  leadership  research  and  leadership  training  development. 

As  Director  of  HumRRO  Division  No.  U,  he  is  presently  responsible  for 
direction  of  its  research  program  in  motivation,  leadership  and  both  group 
and  individual  effectiveness.  He  is  a  Fellow  of  the  American  Psychological 
Association  and  the  Georgia  Psychological  Association.  He  holds  membership 
in  the  Southern  Society  of  Philosophy  and  Psychology,  Psychometric  Society, 
and  National  Council  on  Measurement  in  Education. 

Mr.  3.  Michael  Berger 

Mr.  Berger  is  currently  Technical  Advisor  and  Data  Systems  Coordinator  of 
the  U.  S.  Amy  Military  Occupationeil  Data  Bank  ()iK}DB).  A  graduate  of 
Syracuse  University  (B.  A.  Psychology,  I960),  Mr.  Berger  served  as  a  case¬ 
worker  with  the  Onondaga  County  Department  of  Social  Welfare  after  one  year 
at  the  Syracuse  University  School  of  Social  Work.  Entering  active  duty  in 
1962,  he  was  eisslgned  as  Personnel  Psychologist  and  then  Recruiting  Officer 
at  the  Albany,  New  York  Recruiting  Main  Station.  Transferring  to  Military 
Intelligence  in  196U,  he  served  as  a  unit  commander  in  Korea  and  then  as 
Chief  of  Personnel  Security  Investigations  at  an  M.  I.  Group  in  Washington, 

D.  C. ,  having  been  school  trained  as  an  Intelligence  Research  Officer  and 
Area  Intelligence  Officer.  Mr.  Berger  came  to  the  Office  of  Personnel  Opera¬ 
tions  as  a  civilian  in  1966  and  was  assigned  as  Chief,  Selection  and  Classifi¬ 
cation  Tests  Section  in  Personnel  Management  Development  Office.  He  moved  to 
Military  Occupational  Specialty  Branch  as  a  management  analyst,  and  in  mid 
1972  assmned  his  current  position.  As  a  reserve  Officer,  Mr.  Berger  has 
served  on  the  facxilty  of  the  First  U.  S.  Amy  Area  Intelligence  School,  Fort 
Meade,  Maryland.  He  is  currently  a  unit  commander  and  serves  as  senior  Amy 
Information  officer  for  the  District  of  Columbia  Amy  National  Guard. 


Dr.  Vincent  P.  Cierl 

Dr.  Cieri  is  currently  Educational  Advisor  of  the  US  Amy  Signal  Center  and 
School,  Fort  Moiunouth,  New  Jersey.  A  graduate  of  Columbia  University  (Ed.D. 
1955),  Dr.  Cieri  formerly  served  as  Director  of  Research  and  Measxirement  for 
the  Union  City,  New  Jersey,  Public  Schools;  as  Director  of  the  Proficiency 
Test  and  AncQysls  Agency  and  the  Chief  of  the  Evaluation  and  Educational  Re¬ 
search  Division  at  the  Signal  School;  and,  as  Technical  Director  of  the  Com¬ 
puter  Assisted  Instruction  Project  at  Fort  Monmouth.  He  also  recently 
served  as  Interim  Product  Manager  for  the  Coiqputerlzed  Training  System  Project 
au  the  Signal  School  for  the  US  Continental  Amy  Command.  He  is  a  member  of 
several  professional  organizations  including  the  American  Educational  Research 
Association.  He  is  edso  a  member  of  the  Graduate  Faculty  of  the  Department  of 
Teacher  Education  at  Monmouth  College. 


LTC  William  R>  Muim 


LTC  Muxin  is  currently  Special  Assistant  to  the  Assistant  Comiaandant,  US 
Amy  lingineer  School,  Fort  Bel  voir,  Virginia,  A  graduate  of  Ohio  State 
University,  LTC  Muon  has  served  in  a  variety  of  school,  staff  and  eomnand 
assignments.  School  assignments  include  a  prior  tour  at  the  Engineer 
School,  Fort  Bel  voir  and  the  US  Amy  Engineer  School  Europe,  Recent 
assignments  include  duty  with  the  Office,  Chief  of  Engineers,  Department 
of  the  Amy  and  Commander  39th  Engineer  Combat  Battalion  Vietnam.  LTC 
K'onn  has  attended  the  Engineer  Officers  Advanced  Course,  US  Amy 
Engineer  School,  Fort  Bel voir,  Virginia  and  the  Armed  Forces  Staff  College, 
Norfolk,  Virginia, 

Chaplain  (LTC)  John  S,  Snyder. 

Chaplain  Snyder  is  Chief,  Evaluation  and  Methods  at  the  US  Amy  Chaplain 
School,  Fort  Hamilton,  New  York.  A  graduate  of  Princeton  Theological  Semi¬ 
nary  (BD  1957»  ThM  1965 )»  Chaplain  Snyder  is  currently  studying  at  New  York 
Theological  Seminary  and  Rutgers  University.  He  is  a  member  of  the  American 
Personnel  and  Guidwce  Association.  Entering  the  Amy  in  1957,  Chaplcdn 
Snyder's  assignments  have  included  Fort  Kxiox,  Okinawa,  Fort  Monmouth,  Berlin, 
and  Vietnam. 

MAJ  Harold  L,  Shortnagy 

MAJ  Shortnacy  is  currently  Chief,  Evaluation  Division,  Office  of  the  Director 
of  Instruction,  the  US  Amy  Infantxy  School,  Fort  Banning,  Georgia.  A  grad¬ 
uate  of  the  University  of  Nebraska  at  Omaha  (BGS,  1969),  MAJ  Shortnacy  was 
a  graduate  student  at  Kansas  State  University  concurrently  while  cos^leting 
the  US  Amy  Command  and  General  Staff  College  Course  at  Port  Leavenworth, 

Kansas  (l97l)*  His  militeuy  experience  includes  the  normal  command  and  staff 
assignments  in  military  \mits  and  two  tours  as  a  member  of  the  staff  of  the 
Infantry  School.  During  his  second  tour  in  Vietnam  (1969-1970),  he  served 
as  Chief,  Plans  Branch,  Training  Center  Division,  Training  Directorate,  In 
that  capacity  he  directed  the  efforts  of  a  study  group  formed  to  determine 
training  requirements,  funds,  and  facilities  required  to  train  the  Viethamese 
Armed  Forces  during  a  5-year  period  for  submission  to  the  Joint  Chiefs  of  Staff. 

MAJ  Patrick  H,  Gonaan 


MAJ  Forman  is  currently  Chief,  Command  and  Leadership  Branch  at  the  US  Amy 
Transportation  School,  Foxt  Eustis,  Virginia,  a  position  he  has  occi:gpied 
since  July;  1972.  A  1959  graduate  of  Rutgers  University,  where  he  majored 
in  PoliticcQ  Science,  MAJ  Gorman  has  served  in  varied  command  and  staff  po¬ 
sitions  in  CONUS  and  overseas  since  eatxy  on  active  duty.  Militaiy  schooling 
includes  Infantry  Officer  Basic  Course,  Basic  Airborne  Course,  Transportation 
OrTicers  Advanced  Course,  US  Amy  Command  and  General  Staff  College,  and  the 
Defense  Languate  Institute  (LOATIAN).  MAJ  Gorman  also  attended  a  3-month  course 
in  Creative  Leadership  at  the  Center  for  Creative  Leadership  GreensL^ro,  N.C. 
(Sep  -  Dec  71). 
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Mr.  JaacB  L.  Sherrill 


Mr.  Sherrill  received  the  BA  (1952)  und  MA  {19^5)  degrees  from  George 
Peabody  College,  and  recently  (1971}  completed  a  year  of  graduate  study 
In  Instructionad  Systems  Technology  at  Indiana  IMiverslty.  After  three 
years  of  Hl£^  School  Teaching,  Mr.  Sherrill  Joined  the  US  Amy  Aviation 
School  as  an  Instructor  In  the  Instructional  Methods  Division  in  1953.  In 
i960  he  transferred  to  HQ  USCOHARC,  Office  of  the  Deputy  Chief  of  Staff  for 
Individual  Training  as  a  Staff  Training  Specialist  and  in  1963  he  trans¬ 
ferred  to  the  US  Anqy  Adjutant  General  School  in  his  present  position  as 
Special  Assistant  to  the  Commandant  and  Educational  Advisor. 


TRAIHZliG  QUALir:  COHTROL  WORKSHOP 
ABSTRACTS  OF  PRESENTATIOHS  AHD 
SUMMRIES  OF  OISCUSSKV 

1.  External  Training  Quality  Control 

a.  Abstract  of  "ibe  Militaiy  Cccupactioncd  Data  Bank.”  Current  de¬ 
velopments  vithin  the  MODS,  availaibllity  of  <lata  to  various  users,  and 
usefiOness  of  ^K)DB  information  at  the  service  schools  for  systens  en¬ 
gineering  and  as  a  supplement  or  replacement  for  the  Post-Graduate  Ques¬ 
tionnaire  Program.  Discussion  of  the  addition  of  data  analysis  to  the 
MODS  operation,  conversion  of  the  measurement  scale  from  frequency  to 
relative  time  spent,  and  some  of  the  findings  resulting  from  data  an¬ 
alysis.  Inqiact  on  POX  and  Post  Graduate  Questionntdre  Programs  was 
also  discussed. 

b.  Abstract  of  "Job  Eveauation  Questl<xinaire  Program  at  the  US  Any 
Signal.  Center  and  School.”  Topics  discussed  included:  rationale  for 
the  program,  control  conditions,  and  some  of  the  mechanics  and  findings 
of  the  program.  A  short  TV.  presentation  used  to  orient  students  was 
shown. 

c.  Discussion  following  the  presentations  centered  on« 

(1)  The  vedue  of  task  frequency  Information  vs  relative-time  spent 
information . 

(2)  The  validity  of  the  MCDB  information, 

(3)  Impact  on  field  of  various  surveys;  MODE  surveys,  school-con¬ 
ducted  post  graduate  questlonncdres ,  and  other  one-time  mailed  surveys. 

(U)  Possible  duplication  of  effort  between  the  MODE  and  post  grad¬ 
uate  questionnaires. 

d.  No  conclusions  or  recommendations  resulted.  Strong  differences  of 
opinion  were  evident. 

2.  Student  Covirse  Critique  Panel. 

a.  The  use  of  a  student  course  critique  panel  was  demonstrated 

using  advanced  officer  course  students  furnished  by  the  Air  Defense  School. 

b.  In  the  discussion  following,  severed  School  representatives  re¬ 
ported  they  were  using  veurious  versions  of  the  techniques  and  consensus 
was  reached  that: 
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(1)  The  technique  offered  an  t.dvemtage  over  the  more  typical 
end-of-course  written  comnents;  naioely,  the  opportunity  to  follow-up 
in-depth  student  eoiniaents. 

(2)  The  technique  was  a  worthwhile  supplement  to  other  student 
course  critique  means. 

(3)  The  criticisms  offered  were  quite  similar  to  those  offered  by 
advanced  course  students  in  other  service  schools. 

3.  EvcLLuation  of  Leadership  and  Communicative  Skills. 

a.  A  panel  composed  of  representatives -of  five  Army  Service  Schools 
and  one  outside  training  consultant  discussed  the  evaluation  of  leader¬ 
ship  and  commiinicative  skills.  Each  participant  submitted  a  paper  which 
vaa  not  delivered  during  the  workshop  but  which  were  made  part  of  the 
record  of  the  workshop  proceedings.  Abstracts  follow  of  each  of  these 
position  papers,  which  reflected  the  personal  views  of  only  the  author. 

(1)  LTC  Munn,  USAES.  Useful  evaluation  of  leadership  skills  can 
and  must  be  acconpllshed  peurticularly  in  beisic  professional  courses.  A 
new  leadership  evsLlustlon  system  currently  being  implemented  at  the  USAES, 
was  described. 

(2)  Chaplib.1  (LTC)  Snyder,  USAGES.  A  description  was  given  of  the 
recently  introduced  evaluation  system  at  the  US  Amy  Chaplain  School  to 
measure  personality  strengths  and  weeUmesses.  Major  principles  are  the 
use  of  descriptive,  non-normative  ratings  by  peers  and  the  systematic  de¬ 
termination  for  specific  Jobs  of  appropriate  affective  skills. 

(3)  MAJ  Shortnacy,  USAIS.  At  USAIS,  considerable  progress  has  been 
made  in  the  objective  evaluation  of  soft  skills.  However,  certain  aspects, 
especially  in  the  leadership  cures,  continue  to  present  a  challenge. 

(4)  MAJ  Gorman,  USATSCH.  A  movement  away  from  a  "trait-characteristics 
of  leadership"  and  towards  a  behavioral  approach  to  leadership  training  in 
Amy  Service  Schools  is  evident.  However,  a  serious  examination  of  how  to 
evaluate  leadership  training  has  not  been  attenpted. 

(^)  Dr.  Jacobs,  HumRRO.  Ibe  paper  discussed  problems  in  the  develop¬ 
ment  of  leadership  evaluation  methods,  within  the  context  of  soft  skills 
systems  engineering  itself.  Problems  include  the  difficulty  of  finding  a 
"true  e35>ert,"  the  generca  lack  of  clcurlty  concerning  ultimate  criterion 
meiwures,  and  the  fact  that  leadership  skills  are  essentially  disjunctive 
(as  most  other  soft  skills  probably  are).  The  paper  agrees  with  Whitmore 
as  to  the  value  of  the  behavioral  scientist,  in  soft  skills  systems 
engineering,  but  also  cautions  against  Ca)  theoretical  biases  that  m^ 
lead  to  misdirected  work,  and  (b)  misconceptions  due  to  lack  of  exper¬ 
ience  as  a  line  executive.  Either  may  be  counterproductive. 
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'6}  Mr.  Sherrill,  USAAG8.  A  review  of  selected  literature  sugflests 
that  present  Anqr  Seridce  School  evaluation  practices  in  leadership  and 
cooBtunlcatlon  skill  training  areas  do  not  possess  sufficient  objectivity 
and  are  not  predictive  of  relevant  criteria.  Ihe  suggestion  is  made  that 
Inproveoents  may  be  possible  through  an  "operationalizing"  approach. 


TRAINING  QUALITY  CONTROL  WORKSHOP 
THE  MILITARY  0CCUPATI(»1AL  DATA  BANK  (MODB) 
HR.  B.  MICHAEL  BERGER 


The  year  1972  presented  a  significant  challenge  to  the  U.  S.  Army  Military 
Occupational  Data  Bank.  Faced  with  a  lack  of  acceptance  and  use  of  MODB 
output,  both  at  the  service  school  level  and  within  the  DA  staff,  the  MODB 
initiated  a  massive  reorientation  of  its  operation  to  establish  itself  as 
a  useful  and  viable  management  tool .  Early  in  1972  it  seemed  that  a 
proposed  system  (MODB  II)  would  provide  the  users  with  just  such  a  tool. 

MODB  II  had  been  discussed  at  a  series  of  user  conferences  and  had  gained 
significant  acceptance.  Within  the  first  quarter  of  the  calendar  year, 
however.  Department  of  the  Army  was  appris^  of  a  joint  occupational 
analysis  system  idiich  seemed  headed  for  adoption  by  all  the  services.  The 
joint  system,  basically  a  comprehensive  series  of  report  production  programs, 
did  not  have  the  capacity  to  handle  MODB  variety  and  production  requirements, 
nor  did  it  contain  a  system  for  questionnaire  development  or  field  data 
collection.  Analysis  did,  however,  reveal  that  the  joint  system  could,  when 
added  to  improved  and  modified  MODB  programs,  provide  an  expanded  occupa¬ 
tional  data  analysis  capability  for  the  Army.  By  mid-1972  it  had  been 
decided  that  the  Army  would  abandon  MODB  II  development  and  concentrate  the 
data  bank  effort  toward  incorporating  the  new  methodology  into  the  MODB 
system. 

Development  of  the  new  system  required  a  substantial  reorganization  of 
data  collection  and  interpretation  methodology.  An  entirely  new  question¬ 
naire  was  required  in  order  that  the  Army  system  be  compatible  with  the 
input  requirements  of  the  joint  computer  facility.  The  new  questionnaire, 
incorporating  the  best  features  of  MODB  I,  many  of  the  proposed  features  of 
MODB  II,  and  concepts  gleaned  from  Air  Force  and  Navy  occupational  question¬ 
naires,  was  prepared.  Although  similar  in  style  to  current  MODB  Question¬ 
naires,  the  redesigned  instrument  contains  certain  new  features  worthy  of 
mention.  The  Background  Section  has  been  greatly  expanded  to  capture  more 
information  about  the  MOS  incumbent  completing  the  questionnaire.  The 
additional  data  will  enhance  the  usefulness  of  the  occupational  information 
by  permitting  greater  variety  in  selecting  data  samples.  Negative  (or  "NO") 
responses  have  been  dropped  from  the  Task  and  the  Equipment  Sections.  In¬ 
cumbents  will  respond  to  and  rate  only  those  items  they  actually  perform, 
use,  or  maintain.  The  most  significant  change,  however,  is  in  the  task 
rating  scale.  Research  conducted  by  our  sister  services  clearly  indicated 
that  "time"  rather  than  "frequency"  was  a  more  valid  measure  of  task  per¬ 
formance.  Whereas  previous  Army  questionnaires  have  measured  tasks  in 
terms  of  frequency  of  performance,  the  new  questionnaire  will  use  a  rating 
scale  based  upon  relative  time  spent  performing  a  task  in  relation  to  all 
other  tasks  performed.  Air  Force  and  Navy  experience  indicates  that  a 
"time  spent"  scale  will  be  of  far  greater  significance  in  analyzing  tasks 
and  developing  programs  of  instruction,  and  will  be  of  greater  value  in 
conducting  interaction  between  occupational  data  and  the  wide  variety  of 
management  systems  upon  which  such  data  impacts. 
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The  new  system  stimulated  imnediate  establishment  of  an  in-house  data 
analysis  capability.  As  a  part  of  the  expansion  and  reorganization  of 
the  data  bank  a  separate  Data  Analysis  element  has  been  created.  Ibis 
eleiiiv.nt  is  presently  operational  under  the  current  MODB  system  producing 
output  in  the  form  of  analytical  reports  for  use  by  the  service  schools 
and  DA  staff  elements. 

To  understand  what  the  MODS  can  do  for  the  user,  the  user  must  understand 
just  what  the  Data  Analysis  does.  Data  analysis  is  nothing  more  than  the 
application  of  a  concentrated  analytic  effort  to  MODB  output  to  determine 
the  impact  of  occupational  data  upon  the  various  management  systems  of 
the  Army.  These  management  systems  inclxuie  doctrine,  budget,  force 
structure,  selection  and  classification,  training,  and  assignment  and 
utilization,  and  can  further  extend  to  the  research  and  development  effort 
by  supporting  studies  conducted  by  agencies  such  as  HUMRRO,  USACATB,  and 
BESRL.  In  one  form  or  another,  MODB  output  has  had  some  impact  on  each 
of  these  systems. 

Basic  data  analysis  begins  with  a  review  and  consolidation  of  the  hand 
written  comments  included  by  MOS  incumbents  on  the  questionnaire  Job 
Evaluation  Page.  These  comments,  which  have  included  remarks  by  as  many 
as  75%  of  incumbents  responding  to  a  single  questionnaire,  are  reviewed 
for  clues  to  significant  problem  areas  and  other  "keys"  to  evaluation 
of  the  computerized  data.  A  rough  report  is  prepared  from  these  comments 
to  be  included  later  with  the  tabulated  data  or  sent  under  separate  cover 
to  the  users.  The  computerized  questionnaire  data  is  studied  to  determine 
trends,  problem  areas,  and  other  items  of  significance  which  might  denote 
potential  deficiencies  in  any  of  the  systems  mentioned.  An  analytical 
report  is  prepared  for  written  or  oral  presentation  to  the  various  users. 
Presentations  have  been  made  to  users  such  as  Army  service  school  staff 
members,  the  Enlisted  Personnel  Directorate,  and  DCSPER  DA  special  staff 
elements  such  as  the  Office  of  the  Provost  Marshall  General. 

During  the  ana''ytical  process,  the  MODB  staff  takes  full  advantage  of 
all  available  information  sources  such  as  the  Enlisted  Master  Tape  File, 
the  Army  Materiel  Plan,  POI  files,  and  force  structure  reports  to  insure 
that  within  the  limits  of  its  capability  its  recommendations  are  coiq)lete 
and  timely.  Analytical  depth  is  limited  only  by  practical  constraints 
imposed  by  personnel  limitations  and  scheduling  requirements  tdiich  force 
a  heavy  workload  on  the  organization.  As  a  result,  the  MODB  relies 
heavily  upon  the  various  users  to  persue  furnished  "clues"  to  their 
logical  conclusion.  In  each  case  the  final  outcome  of  the  analytic  effort 
depends  heavily  upon  action  by  the  recipient  of  the  MODB  report.  Although 
the  MODB  is  constantly  available  to  assist  in  interpretation  and  analysis 
of  its  data,  and  encourages  management  actions  to  correct  deficiencies  and 
improve  the  various  systems,  the  organization  itself  is  not  empowered  to 
direct  accomplishment  of  tasks  regardless  of  their  validity. 
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In  this  regard,  results  have  been  most  gratifying.  In  the  recent  past, 
users  have  applied  the  one  aspect  that  cannot  be  conveyed  through  MODB 
reports  -  JUDGE>ENT  -  and  recognized  the  significance  of  numerous  findings 
through  initiation  of  major  changes  and  revisions  to  existing  policies  and 
structures.  The  MODB  recognizes  that  this  same  aspect  of  judgement  may 
also  determine  that  MODB  findings  are  not  valid,  and  in  fact  incorrectly 
reflect  a  specific  situation  in  the  field,  however,  it  is  felt  to  be  in 
the  best  interest  of  the  Soldier,  and  certainly  the  Army  as  a  whole,  to 
surface  any  and  all  clues  or  other  indicators  as  they  are  discovered 
through  analysis. 

It  is  clear  that  the  MODB  contribution  to  its  users  is  well  worth  the 
possibility  of  occassional  misinterpretation  which  can  and  will  occur 
through  the  analysis  of  data  collected  from  the  field. 


JOB  EVALUATION  QUESTIONNAIRE  PROGRAM 

AT  THE 

US  ARMY  SIGNAL  CENTER  AND  SCHOOL 
Dr.  Vincent  P.  Cieri 


Feedback  has  long  been  recognized  as  an  integral  and 
essential  part  of  the  training  process.  Properly  defined,  it 
represents  the  return  to  a  control  center  of  the  information 
regarding  events  under  its  control.  Other  psychologists 
define  it  as  the  process  of  sensing  and  correcting.  Prominent 
educational  psychologists  have  conducted  studies  in  the  feed¬ 
back  process  which  show  it  to  be  the  strongest,  moet  important 
variable  controlling  performance  and  learning  and  that  it  is 
most  effective  when  it  is  frequent,  precise,  and  prompt. 

With  these  facts  in  mind,  the  US  Army  Signal  Center  and 
School  has  placed  a  great  deal  of  emphasis  on  conducting  an 
extensive  and  effective  post-graduate  questionnaire  program 
in  order  to  accomplish  the  quality  control  aspect  of  the  systems 
engineering  process.  The  program  is  conducted  within  the 
framework  of  guidance  provided  by  CON  Reg  351-3  entitled 
"Education  and  Training  Administration  and  Training  Policies" 
dated  1  July  1972. 

The  basic  objective  of  our  program  is  to  provide  feedback 
from  the  field  that  can  be  used  to  evaluate  the  effectiveness  of 
school  courses  in  training  personnel  to  perform  the  duties  of 
their  MOS's.  This  evaluation  includes  the  adequacy  of  course 
content,  the  appropriateness  of  training  emphasis,  and  the 
proficiency  of  course  graduates  as  demonstrated  after  a  period 
of  job  experience.  The  information  is  utilized  not  only  to 
furnish  the  command  with  data  about  the  overall  adequacy  of 
school  training,  but  for  use  in  a  very  practical  way  by  curriculum 
specialists  for  updating  programs  of  instruction  and  for  correcting 
training  deficiencies. 


Our  program  has  been  conducted  for  more  than  15  years, 
but  the  techniques  employed  in  the  earlier  years  did  not  yield 
useful  results  primarily  because  of  sparse  returns.  Recent 
modifications  in  procedure  have  increased  the  efficiency  of  the 
program  to  a  point  where  we  are  currently  receiving,  for  some 
courses,  as  high  as  70%  return  from  the  field.  Improved 
methods  and  a  high  degree  of  command  emphasis  are  the  major 
factors  contributing  to  the  current  success  of  the  program. 

The  basic  data  gathering  elements  in  the  program  are  a 
pair  of  questionnaires  for  each  officer  and  enlisted  course--one 
for  the  graduate  and  one  for  his  immediate  supervisor.  Special 
attention  has  been  given  to  making  these  instruments  attractive 
as  well  as  complete  and  at  the  same  time,  easy  to  fill  out. 

They  are  printed  to  insure  attractiveness  and  can  be  answered 
simply  by  checking  a  blank  or  circling  a  number.  All  items 
in  bo^  questionnaires  are  coded  and  the  data  is  punched  into 
cards  for  analysis  by  ADP  equipment 

The  graduate's  questionnaire  is  designed  to  yield  a 
comprehensive  description  of  the  technical  job  behavior  of 
the  graduate  after  he  has  had  3-5  months  experience  in  the 
specialty  for  which  he  was  trained.  The  questionnaire  content 
is  job  oriented.  Detailed  information  is  requested  regarding 
the  job  tasks  performed,  the  frequency  of  performance  of  these 
tasks,  and  whether  the  jobs  are  difficult.  The  same  information 
is  requested  for  equipment  used  or  maintained,  and  regarding 
techniques  or  procedures  employed  in  performing  these  tasks. 
Space  is  provided  for  the  respondent  to  list  equipment  or  tasks 
not  covered  by  the  printed  questionnaire.  Supplementary 
information  useful  f«.r  evaluating  job  performance  is  also 
requested  and  includes  the  graduate's  primary  MOS,  duty  MOS, 
level  of  maintenance  performed  and  type  of  unit  to  which  assigned. 

The  supervisor's  questionnaire  includes  questions  about 
the  graduate's  duty  assignment,  on-the-job  training,  as  well  as 
a  rating  of  the  graduate's  job  proficiency  in  major  task  areas. 

For  this  latter  purpose,  an  ei^>t-point  scale  furnishing  a 


measure  of  job  proficiency  in  each  task  and  skill  or  knowledge 
area  as  well  as  an  overall  rating  is  used.  Copies  of  these 
questionnaires  are  available  to  you  if  you  wish  to  examine  them 
in  detail. 

These  questionnaires  are  forwarded  to  the  graduate  and  to 
the  supervisor  3-5  months  after  the  student  has  graduated.  The 
materials,  with  a  letter  of  transmittal  from  the  school's  Commander, 
are  sent  to  the  commander  of  the  unit  to  which  the  graduate  is 
assigned.  Self-addressed  return  envelopes  are  furnished  to 
facilitate  the  return  of  the  questionnaire  by  the  gradiiate  and  his 
supervisor  independently. 

Several  direct  actions  are  taken  to  encourage  the  graduate 
to  return  his  completed  questionnaire.  One  of  these  is  a  brief 
television  presentation  which  is  presented  to  the  students  two  days 
prior  to  graduation.  Additionally,  during  the  graduation  ceremony, 
members  of  the  graduating  class  are  again  reminded  that  a  question¬ 
naire  will  be  sent  to  them  and  a  personal  appeal  is  made  for  their 
cooperation  in  the  feedback  process. 

A  critical  requirement  for  valid  findings  is  that  the  graduates, 
whose  questionnaires  are  included  in  the  sample  for  analysis,  are 
representative  of  the  population  of  recent  graduates  with  duty 
assignments  in  a  particular  MOS.  Further,  the  duty  assignments 
of  these  graduates  must  be  representative  of  the  varied  types  of 
units  and  geographical  areas  to  which  graduates  are  assigned. 

One  hundred  percent  sampling  of  graduates,  coupled  with  the 
aggressive  program  to  maximize  return  of  the  completed  question¬ 
naires,  furnishes  reasonable  assurance  that  the  first  part  of  this 
requirement  is  satisfied.  Normal  replacement  of  personnel  over 
a  period  of  time  can  be  expected  to  yield  returns  from  all  types 
of  units  and  geographical  areas,  but  the  representative  nature  of 
the  returns  should  be  verified  ^ring  the  analysis  process. 

Sample  size  is  a  consideration  in  any  survey.  Other  con¬ 
siderations  aside,  the  larger  the  sample,  the  more  reliable  the 
results.  About  one  hundred  is  the  minimum  number  of  question¬ 
naires  considered  for  analysis  in  most  instances  while  up  to 
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five  hundred  or  more  may  be  desirable  for  detailed  analysis 
of  sub-samples  to  include  such  areas  as  type  of  unit,  type  of 
job  activity  and  equipment  used. 

A  practical  limitation  to  sample  size  is  the  output  of  the 
course.  If  the  output  is  small,  the  number  of  graduates  may 
not  reach  100  even  over  an  extended  period  of  time.  However, 
if  a  high  percentage  of  returns  can  be  obtained,  so  that  the 
sample  of  analysis  includes  essentially  all  graduates  during  the 
time  frame  in  question,  at  least  the  findings  will  be  based  on 
the  best  available  sample. 

Additionally,  we  must  face  the  fact  that  many  of  the 
returns  will  show  that  some  graduates  are  assigned  to  duties 
outside  of  the  specialty  for  which  they  have  been  trained. 

Thus,  the  sample  may  be  reduced  in  some  instances  to  40-50 
percent  of  the  questionnaires  originally  mailed.  In  these  cases, 
we  have  concluded  that  it  is  best  to  analyze  the  usable  returns 
since  findings  of  a  survey  based  on  a  relative  small  number  of 
graduates  may  be  better  than  no  information  at  all  under  some 
circumstances. 

In  USASCS  courses  having  large  inputs,  100  percent 
sampling  may  yield  the  required  number  of  returns  within  a 
year.  A  time  limit  of  approximately  two  years  is  established 
even  for  low  density  courses  to  insure  that  the  information  is 
reasonably  current  If  the  questionnaires  are  accumulated 
over  an  excessively  long  period,  the  gain  in  reliability  may  be 
offset  by  loss  in  validity  resulting  from  changes  in  the  job 
situation  during  that  period. 

When  a  suffiCA^nt  number  of  questionnaires  is  accumulated 
for  analysis,  they  are  screened  to  insure  that  each  one  is 
complete  and  free  of  discrepancies  that  might  invalidate  the 
findings.  The  principal  requirements  for  inclusion  of  a  question¬ 
naire  in  the  sample  for  the  primary  analysis  are;  First,  the 
duty  M06  of  the  graduate  must  be  the  one  for  which  he  was 
trained,  and  second,  the  period  of  his  duty  assignment  must  be 
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at  least  two  months  but  not  more  than  nine  months. 

Data  are  punched  into  cards  for  processing  by  ADP  equipment. 

In  the  analytical  process  for  the  graduate's  questionnaire,  a  simple 
frequency  count  is  made  of  the  response  to  each  item.  These 
frequencies  are  converted  to  percentages  of  the  relevant  base  number. 
Student  comments  are  also  reviewed  for  additional  information.  A 
similar  procedure  is  used  for  the  supervisor's  questionnaire  except 
for  the  ratings  of  job  proficiency.  In  this  area,  a  simple  frequency 
count  is  made  for  the  ratings  in  each  task  area,  the  frequencies 
being  converted  to  percentages  to  permit  comparison  of  the  distribution 
for  task  areas.  Mean  ratings  are  also  computed  for  each  task  area 
to  permit  comparison  of  areas.  A  weighted  composite  rating  is  then 
computed  for  each  graduate  and  a  frequency  distribution  made  of  these 
overall  proficiency  ratings. 

The  scope  of  the  information  derived  about  job  activity  of 
graduates  is  suggested  by  the  following  list  of  topics  in  one  survey: 

a.  Duty  assignments 

b.  Length  of  time  in  job  since  graduation 

c.  Level  of  maintenance  at  which  graduate  worked 

d.  Major  equipments  maintained 

e.  Troubleshooting  aids  used 

f.  Job  tasks  performed  (including  frequency  of  performance, 
and  whether  the  taste  were  easy  or  difficult  at  the  time  of  the  report) 

g.  Job  taste  performed  and  equipments  worked  on  that  are 
external  to  MOS  for  which  trained 

h.  Job  proficiency  of  graduates 

From  this  detailed  information,  it  is  possible  to  develop  a  com¬ 
prehensive  analysis  of  the  job  activity  of  course  graduates  after 
they  have  been  on  the  job  for  a  substantial  period  of  time.  Com¬ 
parison  of  the  graduate's  job  activities  and  equipment  worked  on 
with  the  program  of  instruction  furnishes  a  basis  for  evaluating 
adequacy  of  course  content.  The  proportion  of  graduates  who  main¬ 
tain  given  equipments  or  perform  other  tasks,  the  frequency  of 
performance  and  the  difficulty  encountered  in  performing  them, 
together  with  added  comments  by  both  graduates  and  supervisors 


furnish  many  insights  into  the  relation  between  school  training 
and  Job  performance  and  requirements. 


The  analysis  may  bring  to  light  training  deficiencies  that 
require  corrective  action  by  the  course.  Occasionally,  evidence 
is  found  of  greater  emphasis  in  the  course  in  a  given  subject 
matter  area  than  is  required  by  the  job  situation.  Findings 
have  also  at  times  revealed  the  prevalence  of  new  equipment 
in  the  field  not  previously  covered  in  the  course  or  the  fact 
that  equipment  no  longer  common  in  the  field  is  still  included 
in  the  course.  Other  findings  have  at  times  referred  to  the 
need  for  strengthening  techniques  and  procedures  such  as 
troubleshooting,  requiring  a  change  in  instructional  emphasis 
in  related  course  areas. 

The  conclusion  that  is  conunon  to  all  surveys  made  to 
date,  however,  is  the  general  excellence  of  the  training  in 
the  USASCS  specialist  and  officer  courses  since  analyses 
have  consistently  shown  that  graduates  can  perform  their 
technical  duties  at  an  acceptable  level  of  competence  after  a 
period  of  on-the-job  training.  Further,  they  have  been  able 
to  a(hipt  their  school  training  to  a  wide  range  of  unfamiliar 
equipment. 

A  comprehensive  written  report  of  the  findings  of  each 
questionnaire  survqr  is  prepared  and  includes  a  narrative 
portion  which  summarizes  the  evaluation  of  the  effectiveness 
of  training  in  the  course.  Detailed  tabular  data  are  also 
furnished  for  use  by  curriculum  specialists  and  course  per¬ 
sonnel  who  may  require  more  specific  information.  The 
written  report  is  furnished  to  the  appropriate  department 
director  for  review  and  comment  to  include  an  indication 
in  these  comments  of  the  action  taken  on  each  conclusion 
reported  in  the  survey.  These  comments  are  then  incorporated 
in  the  published  report  which  is  then  disseminated  to  the 
appropriate  command  and  staff  elements  in  the  school. 
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In  addition  to  comprehensive  reports,  informal  surveys 
a  e  made  from  time  to  time  on  request  from  the  instructional 
departments  for  information  about  specific  equipment  or  about 
job  performance  in  selected  areas.  Completed  questionnaires 
on  hand  at  the  time  of  the  request  are  used  for  these  surveys. 

At  times,  an  informal  report  is  prepared  or  on  other  occasions, 
course  personnel  may  simply  inspect  the  questionnaire  file  for 
data  to  provide  the  information  for  a  specific  problem.  Another 
report,  called  a  Fast  Feedback  Report,  can  be  requested.  This 
includes  a  computer  print- out  of  all  the  graduate  and  supervisor 
data  available  in  raw  form.  These  data  can  be  used  for  fast 
examination  and  analysis  by  course  personneL 

The  graduate  questionnaire  program  which  I  have  briefly 
described  has  played  a  large  part  in  the  development,  revision 
and  reorganization  of  courses  of  instruction  at  the  USASCS, 
furnishing  objective  information  to  support  decision  making. 

The  key  ingredients  in  this  successful  effort  have  been,  in  my 
opinion,  the  strong  Command  support  given  to  the  program,  the 
highly  professional  aspects  of  the  development  of  the  program 
instruments  as  well  as  the  ease  of  administration  for  the 
graduates,  their  supervisors,  and  for  school  staff  personnel. 

These  elements  have  given  the  USASCS  a  feedback  program 
which  has  not  only  been  looked  to  for  guidance  by  the  staff  of  the 
school  but  has  been  supported  and  sought  after  by  the  faculty  in 
the  pursuit  of  a  most  effective  and  viable  training  program  for 
the  school. 
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TRAINING  qUALITY  CONTROL 
USER  REACTION 
NAJ  Richard  F,  Malone 
US  Amy  Adjutant  General  School 

All  of  our  Service  Schools  ask  our  students  to  evaluate  the  training  they 
received*  For  the  most  part  ve  solicit  students  coxnznents  in  vriting.  The 
styles  ve  use  range  all  the  way  from  quite  elaborate  ADP  sunnnarized  Ques¬ 
tionnaires  with  rating  scales  to  very  unstructured  and  open  ended  student 
comment  sheets  containing  8iBQ)le  directions:  e*g«,  ''Submit  any  comments 
you  believe  appropriate." 

One  posisble,  and  little  used,  means  of  obtaining  our  students'  evaluation 
of  their  training  is  to  form,  from  a  random  selection  of  a  class ,  a  panel 
of  five  or  six  students.  Possibly  the  one  major  advantage  of  the  "written 
comments"  approach  offered  by  the  panel  is  the  opportunity  to  follow  up  in 
depth  certain  student  comments.  A  second  possible  advantage:  some  student 
concern  may  be  of  such  a  nature  tha^  the  student  may  be  reluctant  to  commit 
them  to  writing  and  may  be  more  likely  to  submit  them  orally. 

We  will  shortly  demonstrate  the  use  of  this  approach.  Before  we  do,  a 
few  observations  are  appropriate. 

We  suggest  that  only  one  representative  of  the  staff  and  faculty  be  present 
and  that  he: 

a.  Serve  as  the  panel  chairman 

b.  Be  a  member  of  the  DOI  rather  than  one  of  the  instructional  depts. 

c.  Have  rank  conqparable  with  s-cudents  comprising  the  panel 

d.  Play  the  role  of  a  neutral  observer  interested  only  in  obtaining, 
as  nearly  as  is  possible,  the  real  concerns  of  the  students 

A  couple  of  administrative  observations  about  the  demonstration. 

a.  Our  thanks  to  the  US  Amy  Air  Defense  School  for  edloving  us  to 
use  their  students 

b.  Please  treat  the  substance  of  the  students'  comments  as  privilege 
"v  information:  not  to  leave  the  room 

c.  We  should  focus  our  attention  on  the  method:  is  it  worthwhile; 
should  ve  use  the  approach  to  replace  or  supplement  our  more 
usual  procedures? 

The  students  serving  on  this  panel  received  yesterday  a  DF  containing  infor¬ 
mation  about  their  selection  and  the  purpose  of  the  panel,  A  copy  of  these 
instructions  are  at  Inclosure  one. 
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AT. . . . 


Course  Critique 


TO;  Date;  _ 

MAJ  MaloneT 

1.  You  and  fo\ir  others  have  been  selected  to  represent  your  class  in  a  panel 
discussion  formed  to  critique  your  coune.  Please  be  present  in  room  _____  at 
_  on  _  _ . 


2.  You  are  asked  to  be  prepared  to  discuss  and  evcduate  any  part  of  your  course 
you  believe  needs  improvenent  as  veil  as  those  peurts  which  should  remain  unchanged. 
Do  not  limit  your  discussion,  necessarlUy ,  to  the  academic  side  of  your  training. 
Instead,  examine  any  areas,  favorable  or  mfavorable,  which  had  the  greatest  im¬ 
pact  on  the  students  in  your  class.  Belov  are  listed  possible  areas  (as  well  as 
others  of  your  choice)  you  may  wish  to  discuss.  You  may  wish  to  use  the  reverse 
side  to  write  some  of  your  preparatory  notes. 


Administration 


Director  of  Instruction 
Materials 


Inprocessing 

Outprocessing 

Housing 

Pay 


Reference  materials 
Reading  assignments 
Classrooms 
Equipment 
Training  Aids 


Course  Content 


Competency 


(Annexes  in  PCI 
listed  here) 


How  qualified  do  you 
feel  to  perform  on  the 
Job 


Inatruetors 


Evaluation 


Subject  knowledge 
Teaching  techniques 
Instructor/student  interaction 


Tests 

Procedtires 

Policy 


Inclosure  one  to  '^ser  Reaction**. 


i. 
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Training  Quality  Control 
Evaluation  of  Leadership  &  Comnmnicative  Skills 


*  W.R.  Hunn^  LTC 
US  Any  Bigineer  School 


Evaluation  of  leadership  in  an  academic  envlrohment  -  can't  be  done, 
not  valid^^we  don't  even  have  a  good  definition  of  leadership  -  so  go  the 
remarks  cJr  maiy  educators.  I  belong  to  the  minority  group  that  believe 
that  leadership  evaluation  can  be  done.  Admittedly,  the  answers  we 
arrive  at  are  not  as  precise  as  those  in  a  mathematical  equation  but 
exceedingly  better  than  none  at  all.  In  addition,  it  is  my  belief  that 
CONARC  Schools  must  evaluate  leadership  skills,  particularly  in  our 
basic  professional  courses.  Basic  course  students  are,  for  the  most 
part,  untested  leeuiership  products  and  in  the  case  of  a  few  ROTC 
graduates,  do  not  possess  those  leadership  traits  that  aure  required  in 
todays'  professional  Amy. 

The  development  and  evaluation  of  leadership  in  EOBC  students  is  of 
prime  concern  at  the  US  Amy  Engineer  School  (USAES).  The  School  is 
currently  implementing  a  new  leadership  evaluation  system  for  Engineer 
Officer  Basic  Course  students.  This  system  has  been  a  joint  project 
between  the  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI)  and  the  Engineer  School.  The  evaluation  is  accomplished 
by  a  student  leadership  grade  and  development  of  an  overall  leadership 
profile  of  each  student  officer. 

a.  Student  leadership  grades  are  established  from  engineer  platoon 
leader  performance  lype  testing,  physical  fitness  and  peer  and  supervisor 
ratings. 


b.  In  conjunction  with  the  student  leadership  grade,  an  overall 
leadership  profile  will  be  developed  for  each  EOBC  student.  The  develop¬ 
ment  of  this  profile  begins  during  the  first  week  when  each  student  officer 
will  take  the  Officer  Evaluation  Battery  (OEB),  Battery  results  provide 
the  Class  Supervisor  with  information  on  leadership  potential  of  individual 
students  in  the  three  a^’eas  of  combat  leadership,  managerial-technical 
leadership  and  motivation  and  drive.  Information  will  be  utilized 
initially  by  the  Class  Supervisor  in  assignment  of  class  leadership 
positions  and  student  counseling.  The  0KB  will  not  be  utilized  in  the 
determination  of  the  student  leadership  grade  but  does  contribute  to  the 
establishment  of  the  overall  student  leadership  profile.  The  components 
identified  and  evaluated  in  the  leadership  profile  are; 

Combat  Leadership  -  Defined  as  ohe  abilities  and  knowledge  necessary 
to  lead  men  in  conibat  and  the  abili  y  to  interact  in  a  one  to  one 
situation  with  men  and  have  them  "iollow"  example  and  direction. 


*  The  views  expressed  in  this  paper  are  those  of  the  author  and  do  not 
necessarily  reflect  those  of  the  USA  Engineer  School  or  the  Department 
of  the  Amy. 
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Managerial- Technical  Leadershi;?  -  The  knowledge  and  abilities  necessary 
to  organize  and  coordinate  personnel  and  materials  into  an  efficient  and 
effective  structure. 

Motivation  and  Drive  -  Ihe  willingness  to  exercise  and  utilize 
capabilities  to  tiieir  i'ullest. 

Scholastic  Engineering  Potential  -  The  ability  to  grew  intellectually 
and  process  educationally  as  an  engineer  to  meet  the  requirements  of  the 
future. 

Physical  Fitness  -  The  physical  stamina  required  to  perform  in 
field  situations. 

Additional  Leadership  Experiences  -  Other  e:5)eriences  contributing 
to  the  overall  leadership  development  of  the  student  officer. 

c.  While  the  above  cemponents  are  to  same  degree  related  to  each 
other  they  are  independent  so  that  it  is  not  necessary  that  an  officer 
high  in  one  component  be  low  in  another.  It  is  however,  highly  desirable 
that  graduates  rank  high  in  all  areas.  Ttie  assessment  procedTires  are 
divided  into  two  broad  categcxries,  measures  of  knowledge,  and  chyacteristics 
of  performance.  (Incl  1 )  I&iowledge  tests  will  focus  upon  right/wrong 
indormation  subject  matter  tests  and  right/wrong  performance,  hands-on 
tests .  The  evaluation  of  Performance  Characteristics  will  be  made  by  the 
supervisor,  instructor,  associates,  and  the  sttident  himself  and  will  focus 
upon  the  process  of  performing  a  task  rather  than  the  right/wrong  aspects 
of  the  solution.  The  procedure  used  to  select  measures  for  each  component 
was  rational.  Die  OEB  scores  were  used  as  reference  variables  and  other 
evaluative  elements  were  selected  which  were  correlated  with  the  OEB  and 
each  other.  Die  recommended  wei^ts  assigned  to  each  variable  were  based 
upon  the  known  reliability  and  validity  characteristics  of  the  measures. 

The  benefits  of  the  program  are  that  once  strengths  and  weaknesses  in 
leadership  development  and  potential  are  identified,  counseling  and 
guidance  can  be  given.  This  identification  can  help  the  School  to  plan 
more  effective  programs  and  make  pregraduation  decisions  on  the  capabilities 
of  individual  officers.  It  also  provides  a  longer  range  prediction  of 
the  student  officer  performance  as  a  leader  which  will  be  coordinated 
closely  with  the  Officer  Personnel  Directorate.  In  addition  it  aids  the 
student  officer  to  plan  future  study  and  development  and  to  make  more 
informed  career  choices.  It  is  readily  conceded  that  the  USAES  leadership 
evaluation  system  is  far  from  perfect,  however,  I  believe  it  represents 
a  tremendous  improvement  over  evaluative  systems  formerly  utilized. 
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TRAININCr  QUAUTV  CONTROL  WORKSHOP 


ASSESSING  AFFECTIVE  SKILLS:  POSSIBILITIES  AND  PROBLatS 
Eraluatlon  of  Leadsrahdp  and  ConDunlcatlve  Scllls 
Ch  (LTC)  John  S.  amyder 
U.S.  Am^  Ou^laln  School 


Inr.rodttction 


To  engage  In  the  aesesanent  cf  leader^ip  is  to  face  the  fact  that  thile 
«e  nay  often  agree  uhen  an  individual  is  an  outstanding  leader,  it  is  iaor« 
difficult  to  reach  a  oonaeneus  on  idiat  factors  are  essential  to  leader^:^p. 
Peraonalitlee  differ  and  styles  vaiy  anong  equally  effective  leaders.  Pur- 
theznore,  acne  situations  seem  to  call  for  a  different  sort  of  leadership 
than  other  situations.  A  systematic  laeasure  of  leadership,  therefore,  must 
be  inclusive  and  sophlstieated  enough  to  recognize  the  variations  of  effec¬ 
tive  leadership  behavior,  as  well  as  the  traits  which  sure  clearly  the  most 
desirable  in  particular  Jobs  or  situations. 

This  problem  has  been  sunitely  felt  at  the  Quq>lain  School,  charged  as  we 
are  to  evaluate  the  students  in  the  advanced  chaplain  course  (C-22)  for  ca¬ 
reer  managsmmt  purposes  beyond  a  mere  academic  report.  Acadmic  achievement 
alone  never  has  been  acc^ted  as  en  adequate  measure  of  a  ch^laln's  perfom- 
ance,  and  faculty  and  peer  ratings  traditionally  have  been  used  to  provide  to 
the  Gb^lsdn  Branch  management  data  on  his  leadership  style  and  operating  ef- 
ficienqr.  While  these  kinds  of  ratings  have  never  been  popular  and  are  sub¬ 
ject  to  favorltlaa,  guesswork,  and  generalities,  the  necessity  to  make  some 
kind  of  personallly  assesssnnt  has  not  been  questioned.  In  fact,  with  the 
currant  stress  in  the  Cheplain  School  on  task-oriented  instruction,  the  neces¬ 
sity  for  such  an  evaluation  is  more  clear  than  ever.  Not  only  is  it  necessary 
for  career  planning  purposes,  but  also  it  can  assist  in  meeting  a  fairly  re¬ 
cently  recognised  need  in  the  training  of  cluq^ains  -  learning  and  growing  in 
the  affective  dMsain.  An  individual's  affective  strength,  or  leadership 
quotient,  is  a  coupler  mixture  of  personality  cipaeities  and  traits,  and 
should  be  measured  by  «  systematic  description  of  his  separate  behavioral  com¬ 
ponents. 


Assessswnt  of  Affective  3cills  at  the  Chaplain  School 

Peer  and  faculty  ratings  of  C-22  students  were  discontinued,  not  only  be¬ 
cause  of  the  problaais  mentioned  abxve,  but  because  a  method  has  Iteen  worked 
out  which  offers  a  more  objective  md  oomprehensive  a»asurs  of  leadership  and 
work  styles.  It  should  be  noted  t';at  the  C-22  class  is  divided,  for  aLaost 
all  of  the  instruction,  into  12-bm<x  grorq>s,  each  with  a  faculty  advisor.  The 
close  asaoolation  of  t^se  13  man  makes  possible  a  fairly  precise  measure  of 


each  MBber'  e  behavlorml  stgrle,  as  observed  by  the  other  esiiberB  over  sev^ 
eral  aonthe'  tias.  A  rating  dtoet  is  aade  out  b7  s^h  aan  on  all  the  aaa- 
here  of  the  gxoig>,  using  the  Pertsonalltar  and  PerceptioK  Ihventoxy  (PAFI) 
originated  by  M.M.  Ko stick.  )blle  ay  piuxpose  liere  is  not  to  deKrlbe  the 
PAPI  in  detail,  I  vill  say  that  each  asaber  uses  the  foxa  to  adce  a  descrip¬ 
tive  pictuxe  of  each  of  the  others.  It  is  not  a  nozaatire  sorting,  fhe 
average  rating  by  the  other  assibere  In  the  gzo\g>  constitutes  eadi  student's 
personality  profile,  ehiefa  provides  data  for  career  aaiagasent. 

Because  ee  wanted  to  aake  affective  akUls  count  toward  the  grade 
eazned  b7  the  ebe^laln  students,  a  aethod  was  devised  to  aake  Ihs  affective 
score  count  for  ^  of  the  overall  acadsalc  average.  The  plm  wozks  basic¬ 
ally  by  establidilng  a  list  of  ten  aajor  duq;>laln  jobs,  dstezalnlng  throu^ 
e:q>eriaaeed  job  holders  and  supervisors  idiat  sort  of  peraonalily  profiles 
best  would  satisfy  the  daaands  of  the  various  jobs,  dsteraine  the  sequence 
of  increased  carser  significance  In  the  jobs  (i.e.,  wore  responsibility, 
higher  rank),  and  then  assipi  Increasing  weights  to  those  positions.  The 
students'  affective  scores  are  aatcbed  against  the  job  profiles,  with  those 
aatdklng  the  aore  heavily  weif^ted  jobs  scoring  higher. 


Iheolved  Problaas 


Our  use  of  Kostiek's  PAPI  is  still  in  its  beginning  stage,  and  the  suc¬ 
cess  of  this  aethod  of  evaluating  affective  skills  will  only  be  knoai  after 
several  aonths,  if  then.  Sndi  instroasnts  for  asasurlng  affective  style 
are  notorioualy  unreliable,  ^thou^  the  Inventoxy  has  been  used  extensive¬ 
ly  in  Aaeriean  Indastzy,  we  wonder  about  the  possibility  that  it  will  be 
tranaparant  to  anas  students.  Moreover,  any  asasure  of  affective  akills  In 
a  C-22  coiurse  is  handlo^pped  by  jhe  fact  that  the  environasnt  is  a  school 
rather  than  the  job.  Mhatever  the  students'  job  status  or  level  of  respon¬ 
sibility  has  bean  before  ooaing  to  the  school  for  the  career  oourae,  tb^ 
tend  to  revert  In  part  to  adolescent  patterns  of  student  behavior.  How  dif¬ 
ferently,  we  must  aak,  does  a  aan  operate  in  his  present  student  grotg>,  con- 
parsd  to  his  behavior  as  a  brigade  dug>laln  or  In  soas  other  responsible 
position?  Finally,  there  is  the  problea  of  the  validity  of  this  sort  of 
evaluation.  Does  a  hij^  affective  score  correlate  positively  with  success 
in  jobs  with  hl^er  career  aLgiifieance?  ¥b  don't  know  that  yet,  and  obvi¬ 
ously  we  can't  until  aom  sort  of  longitodlnAL  stu^y  is  aads. 


Cwi3ltt»loaB 


toe  pzoblaRS  in  the  precise  end  objective  evalnatijoD  of  affective 
ddlls  axe  evident,  bat  there  are  possiblLltiee  for  effective  neasuree, 
both  for  etinnlati^  learning  and  for  detemining  normative  grading. 

The  verdlot  ie  not  yet  in  on  the  eaccese  of  the  experiineatai  method  in 
use  at  the  Chaplain  School.  It  xs,  hoeever,  a  egrstemauie  attempt  to 
evaluate  leaderabip  end  other  Intezpersonel  akUls  ae  they  pertain  to 
the  %>eeifle  Jobs  for  ataldi  the  stoident  is  being  pr^red.  By  getting  a 
desc^tlve  profile  from  all  the  msmibers  of  his  instructional  group,  a 
group  mllh  iMeh  he  has  been  intimately  involved,  me  hope  that  ee  axe 
getUng  as  objective  and  accurate  a  plotare  of  his  behavior  as  is  possi¬ 
ble  with  todiy' s  methods.  Regardless  of  the  success  or  failure  of  the 
Ohigtlain  School's  use  of  the  PiPI,  however,  two  aqoects  of  this  ^roach 
Miould  rwsain  valid  and  be  relevant  to  leadsrahip  assesmwnt  in  other 
sdhoole:  the  use  of  descriptive,  non-noxmative  ratings  b7  peers  and  the 
qrstamatic  determination  for  specific  jobs  of  ^ropriate  affective 
dklUs. 


TRAIWrUG  QUALITY  CONTROL 

Evaluation  of  Leadership  and  Communicative  Shills 
MAJ  Patrick  E,  Gorman 
US  Anq/  Transportation  School 


The  recent  wave  of  hl^  level »  con^rehensive  studies  evaluating  the 
effectiveness  of  Aray  leadership  has  focussed  unprecedented  attention  on 
this  coB^>lex  interactive  process. 

As  the  mushrooming  reseaurch  of  behavioral  scientists  finds  Increasing  ac¬ 
ceptance  aaoong  top  Department  of  the  Axw  policy-makers,  the  philosophy 
of  mllltaxy  leadership  Instruction  has  also  changed  significantly  to  re¬ 
flect,  for  one,  the  ascendance  of  the  behavioral  over  the  traitlst 
approach. 

Throu^out  the  Amy  Service  School  system  leadership  programs  of  Instruc¬ 
tion  have  undergone  major  revisions  to  deeophaslze  the  traditional  laundry 
list  concept  einbodled  In  earlier  dlscoxirses  on  traits,  principles  and  the 
leadership  concept.  In  favor  of  the  behavioral  approach  which  describes 
the  leadership  process  as  a  dynaoLc  Interaction  deriving  Its  In^etus  from 
individual  needs  and  motivations. 

However,  while  the  educationed  philosophy  underlying  leadership  instruction 
is  changing  little  serious  thought  seems  to  have  been  given  to  redefining 
instruction  objectives  and  means  of  evaluation.  We  continue  in  the  final 
ancQysis  to  regard  leadership  as  an  irreducible  totality  regrettably  beyond 
the  pale  of  effective  measurement,  except  for  a  few  related  peripheral 
shills.  For  example,  two  skills  currently  widely  stressed  are  military 
writing  and  oral  briefings,  which  frequently  serve  as  the  basis  for  evaluating 
an  officer's  communicative  ability.  In  fact  these  one  way  communications 
represent  but  a  small  part,  related  only  in  name  to  the  wider,  more  demanding 
sphere  of  interactive  communications  ("meanings  are  in  people"}. 

I«'s  clear  that  we  have  thus  far  failed  to  come  to  grips  with  the  funda^ 
mental  question  tiltimately  confronting  all  educators  -  what  are  we  trying 
to  teach?  What  is  it  we  want  our  students  to  learn?  What  is  it  we  expect 
them  to  be  able  to  do?  The  issue  of  defining  goals  is  complicated  somewhat 
by  our  inability  to  define  our  own  operation  lexicon.  What  is  the  functional 
distinction  between  "leadership"  and  management"?  Are  they  synonymous  terms 
for  the  same  process,  or  do  they  describe  separate  sets  of  functions  requir¬ 
ing  different  skills? 
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Before  ve  can  establish  realistic,  attainable  cuid  measureable  leadership 
instructioncLl  gocLls,  then,  we  must  first  determine  specific  student  needs 
based  on  an  evaluation  of  field  requirements,  then  identify  those  skills 
the  student  must  acquire  to  satisfy  the  needs.  Although  this  is  an  over* 
8inq>lification,  it  does  hig^li^t  what  has,  to  date,  been  an  area  of  weak¬ 
ness.  This  approach  will  doubtless  load  to  many  other  fields  of  inquiry. 

Are  all  leadership  skills  trainable  in  a  school  environment?  If  not,  how 
do  we  standardize  instruction  on  those  which  are  better  left  to  practical 
application  in  the  field?  .Is  standardization  of  leadership  instruction 
necessary  or  desirable?  Do  the  skills  required  by  an  Infantry  leader 
apply  to  the  Queirtermaster  or  Transportation  leader? 

This  paper  admittedly  contains  no  answers,  but  addressing  the  questions 
posed  is  in  ny  Judgment,  an  essential  prerequisite  to  any  serioias  attempt  to 
measure  effectiveness  of  leadership  Instruction, 


TRAINING  QUALITY  CONTROL  WORKSHOP 
EVALUATION  OF  LEADERSHIP  AMD  COMMUNICATIVE  SKILLS 


By  J.  GERBER 

Evaluation  Divlaion,  DOI 
Fort  Banning,  Georgia 


Leadership  and  comnunicative  skills  are  the  Infantryman's  stock  In 
trade  -  the  peg  upon  which  all  his  other  skills  and  knowledge  hang. 

The  evaluation  of  instruction  in  the  leadership  and  coesRinicatlve 
skills  is,  therefore,  very  Important  to  us  at  USAIS  and  occupies 
a  great  deal  of  our  attention. 

For  the  sake  of  simplicity,  the  material  that  follows  has  been  organised 
along  the  lines  suggested  by  the  title  of  this  panel,  namely  leadership 
and  comnunicative  skills.  By  separating  these  two  areas  for  treatment 
in  this  paper,  I  do  not  intend  to  infer  that  they  are  considered 
separately  as  a  mattei^.  of  routine.  Actually,  I  feel  that  they  should 
be  considered  together  in  the  normal  course  of  events.  That  is  to 
say  that  in  practice,  the  ability  to  comsunicate  is  an  important 
aspect  of  leadership  and  is  normally  one  of  the  factors  upon  which  an 
assessment  of  leadership  (or  supervisory)  ability  is  evaluated.  For 
ease  of  presentation,  however,  I  have  separated  leadership  and 
communicative  skills  in  this  paper. 

At  the  Infantry  School,  we  provide  instruction  in  both  the  explicit 
and  implicit  aspects  of  leadership.  In  the  explicit  aspect,  I  am 
referring  to  such  things  as  factual  information,  principles,  and 
courses  of  action.  In  the  iiiq;>llcit  aspect,  I  am  including  attitudes, 
morals,  values,  and  beliefs.  We  feel  that  we  can  impart  or  teach  the 
explicit  and  influence  the  iiq> licit.  Inclosure  1  shows  a  few  of  the 
explicit  topics,  the  ways  these  topics  are  presented,  and  the  ways  they 
are  evaluated.  Inclosure  2  shows  similar  Information  on  implicit 
aspects  of  leadership. 

As  a  general  rule,  we  believe  that  instruction  must  be  based  on 
specific,  well-defined  training  objectives  in  order  to  achieve  desired 
results.  Under  this  approach,  precise  Instruction  can  be  developed 
and  executed,  and  the  results  can  be  measured  objectively.  Where 
succinct  training  objectives  cannot  be  written,  the  instruction  falls 
into  a  gray  area  of  definition,  and  results  are  difficult,  if  not 
impossible,  to  measure  objectively  given  the  present  state  of  the  art. 
The  instruction  may  well  be  effective,  but  at  the  present  time  we 
don*.t  really  knew  that  it  is  or  why. 
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Leadership  in  the  aggregate  we  can  recognize  after  the  fact.  We 
can  observe  and  note  of  the  man  placed  In  the  position  of  leadership 
that: 


He  did  say  "Follow  me!"  and  the  others  followed,  or 

He  did  wave  an  arm  toward  the  top  of  the  hill  and  the 
others  followed  him. 

The  above  examples  are  after*the-fact,  go/no  go,  observable  phenomena. 

We  can  Infer  that  the  leader  perceived  himself  as  a  leader  and  that 
those  who  followed  him  perceived  him  as  a  leader.  We  aren't  always 
able  to  determine  precisely  why  this  Is  true.  With  even  less  precision 
are  we  able  to  predict  that  this  man  will  follow  that  man  under  any 
and  all  circumstances »  as  some  newspaper  stories  coming  out  of  Vietnam 
have  been  quick  to  point  out. 

Listing  leadership  qualities  or  characteristics  In  a  task  Inventory  Is 
a  very  difficult  imdertaklng.  While  we  might  agree  that  the  "good" 
leader  should  be  honest  and  fair,  and  should  set  the  example  for  his 
followers,  statements  of  these  actrlbutes  are  not  readily  convertible 
to  training  objectives.  Using  situational  type  Instruction,  we  are  able 
to  evaluate  behavior  which  we  Interpret  as  reflecting  honesty  and 
fairness,  for  example,  but  which  may  not  In  fact  do  so.  In  addition, 
ve  do  not  know  to  what  extent  stress  will  cause  what  we  consider  to  be 
proper  attitudes,  beliefs,  etc.,  to  change.  Here  are  some  examples: 

The  young  platoon  leader  who  leads  the  charge  up  a  hill 
may  not  necessarily  believe  that  the  trip  Is  necessary. 

The  Caucasian  comnander  may  demonstrate  statistically  a 
scrupulously  fair  promotion  policy,  yet  hold  a  deep-seated 
contempt  for  non -white  menibers  of  his  conmand. 

The  leader  who  is  truly  fair  and  honest  may  fail  as  a  leader 
because  he  is  not  perceived  as  fair  and  honest  by  his 
subordinates.  Asked  for  specifics,  subordinates  may  not  know 
or  may  not  be  willing  to  state  that  they  are  reacting  to  nonverbal 
cues  such  as  t*e  tilt  of  a  head,  the  flicker  of  an  eye,  or  the 
nqnrlad  other  aspects  of  nonverbal  comnunlcatlon  with  which  the 
person  In  the  leadership  position  la  endowed  or  which  he  affects. 

I  am  saying,  then,  that  we  may  Identify,  teach  toward,  and  evaluate 
affectively  based  behavior  in  the  cognitive  domain  and  delude  ourselves 
into  believing  we  have  embraced  the  affective  domain.  My  frame  of 
reference  here  is  Bloom's  Taxonomy  of  Educational  Objectives.  I  am 
saying  that  as  a  student  enrol lec  in  a  leadership  course,  I  can 
recite  the  School  solution,  apply  the  principles,  demonstrate  mastery, 
and  reject  the  whole  paradigm  in  so  doing. 
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The  task  of  USAIS,  and  indeed  of  all  Service  Schools,  la  to  discover 
more  effective  neams  of  evaluating  the  affective  area  -  the  gut 
feelings  -  resulting  from  the  Instruction  we  present  In  the  so-called 
soft  skills  training,  the  alternative  Is  to  agree  that  we  will 
'ixpose  the  students  to  training  or  Instruction  that  we  hope  will  bring 
about  or  strengthen  attitudes,  beliefs,  morals,  etc.  that  we  believe 
to  be  proper  and  leave  the  judgement  as  to  how  well  we  have  succeeded 
to  those  who  will  observe  and  report  on  the  performance  of  the  students 
In  future  assignments. 

At  this  time,  USAIS  uses  some  objective  testing  of  knowledges  and 
awareness  (cognitive  domain) ,  but  relies  on  subjective  evaluation  of 
the  application  of  knowledge  and  awareness.  Some  of  the  subjective 
techniques  used  are: 

Instructor  use  of  check  list. 

Spot  checks  by  "hard  skills"  Instructors. 

Observations  of  tactical  officers. 

Peer  ratings. 

Counselling  by  successful  leaders. 

Instruction  In  overt  communicative  skills  at  the  Infantry  School  falls 
Into  two  categories:  writing  and  speaking.  We  have  limited  writing 
and  speaking  requirements  In  certain  of  the  basic  level  courses; 
however,  the  Infantry  Officer  Advanced  Course  contains  a  variety  of 
comnunlcatlve  requirements.  Inclosure  3  shows  writing  requirements  and 
evaluation  procedures;  Inclosure  4  shows  the  same  Infonsatlon  on 
speaking  requirements  for  thnt  course.  Although  evaluation  of  these 
requirements  Is  listed  as  subjective,  some  objectivity  Is  Introduced 
through  the  use  of  check-list  type  scoring  for  format,  content,  etc. 

We  are  placing  increased  emphasis  on  the  cooBunlcatlve  skills  In  the 
Advanced  Course.  The  POI  has  been  changed  to  reflect  real-life  require 
ments  which  the  graduates  will  face  In  assignments  subsequent  to 
graduation.  Using  real  life  as  a  criterion,  we  have  eliminated  the 
requirement  to  write  a  monograph. 

At  present,  we  are  studying  the  feasibility  of  allocating  academic 
weight  to  comnunlcatlve  skills  requirements  to  further  upgrade  this 
area  of  Instruction.  Our  feeling  is  that  the  ability  to  speak  and 
write  well  are  Indispensable  tools  of  leadership,  and  the  fact  that 
they  become  Increasingly  laq>ortant.  In  our  view,  as  the  leader 
progresses  up  the  ladder  has  caused  us  to  concentrate  our  emphasis 
on  upgrading  this  area  In  the  Advanced  Course. 

In  summary,  we  at  USAIS  believe  chat  progress  Is  being  made  In  the 
evaluation  of  soft  skills,  but  there  are  some  challenges  In  this 
area  that  we  have  not  solved.  Although  we  are  pleased  at  the 
progress  In  this  area,  we  will  continue  to  apply  diligent  effort  to 
the  resolution  of  the  portions  that  remain  unsolved. 


Incl  1  to 
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SOTJRCE:  INFANTRY  OFFICER  ADVANCED  POI,  21  NOV  72 


TRAINING  QUALITY  CONTROL 
The  Evaluation  of  Leadership  Skills 
T.  0.  Jacobs* 

HumRRO  Division  No.  4 

The  primary  purpose  of  this  paper  is  to  discuss,  to  some  small  extent, 
problems  Involved  in  tRi^valuation  of  leadership  skills.  While  the  topic 
of  the  panel  as  a  whole  concerns  both  leadership  and  comaunication,  the 
latter  area  is  well  addressed  elsewhere.  In  addition,  as  will  become  evi¬ 
dent,  it  will  not  be  possible  to  discuss  evaluation  of  leadership  skills 
without  first  discussing  the  current  state  of  the  art  in  their  identifica¬ 
tion  and  development. 

SOFT  SKILLS  SYSTEMS  ENGINEERING? 

This  frankly  provocative  title  was  selected  to  draw  attention  to  the 
probable  source  of  current  problems  in  quality  control  of  soft  skills 
training.  A  basic  problem,  at  least  insofar  as  leadership  is  concerned, 
is  that  the  systems  engineering  of  leadership  training  is  only  beginning 
to  emerge  from  the  dark  ages.  As  this  paper  will  show,  there  is  a  substan¬ 
tial  lack  of  agreement  on  at  least  the  following  points: 

a.  What  leadership  is. 

b.  What  affects  leadership  in  organizations. 

c.  The  training  experience  that  will  produce  improved  leadership. 
d«  Satisfactory  measures  of  leadership  ability. 

Other  participants  in  this  conference,  and  in  previous  conferences,  have 
addressed  themselves  to  the  problems  of  soft  skills  systems  engineering. 
This  paper  consequently  will  not  belabor  the  point  —  and  the  difficulties 
that  have  been  encountered.  A  brief  examination  of  two  sources  will  serve 
as  an  adequate  review  of  the  problem,  to  place  the  following  discussion  in 
perspective. 

Crawford  (1966)  has  described  seven  steps  in  the  development  of  training. 
These  steps  are  shown  in  Figure  1. 


*The  views  expressed  in  this  paper  are  those  of  the  author  and  do  not 
necessarily  represent  the  official  opinion  or  policy  of  the  President, 
Human  Resources  Research  Organization  or  the  Department  of  the  Army. 


Steps  in  the  Bevel opaent  ef  Training 


Figure  1 
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A  case  can  be  made  that  the  major  problem  In  soft  skills  systems  engineering 
lies  In  the  second  step,  analysis  of  the  particular  job.  In  his  discussion 
of  this  step,  Crawford  says,  "We  are  Interested  In  the  Inputs  to  the  job  — 
the  kind  of  stimuli  and  requirements  placed  on  the  individual  and  the  out¬ 
puts  from  the  job  —  what  the  man  does  to  bring  about  system  output."  This, 
of  course.  Is  a  reasonable  assertion.  The  problem  arises  when  one  attempts 
to  Implement  this  step.  In  falmes.'],  Crawford's  paper  Is  not  intended  to  be 
a  "how- to-do- it."  Nonetheless,  it  ..s  instructive  to  note  that  a  rough  count 
shows  that  only  91  words  were  allocated  to  explication  of  this  step.  But, 
remeiid>er  that  Crawford's  paper  was  not  a  "how-to-do-it"  guide. 

To  a  much  greater  extent,  CONARC  Regulation  350-100-1  was  Intended  to  be 
just  that.  Again,  the  purpose  of  this  paper  is  not  to  malign  the  existing 
procedures.  Nonetheless,  examination  of  that  section  of  the  CONARC  Regula¬ 
tion  pertaining  to  development  of  the  task  Inventory  shows  that  only  five 
pages  of  a  total  of  49  pertain  to  this  crucial  portion  of  the  systems  engineer¬ 
ing  effort. 

TWo  observations  can  be  made.  First,  this  level  of  guidance  probably  in¬ 
dicates  that  the  writers  of  this  regulation  did  not  anticipate  that  this 
step  would  be  as  much  a  problem  in  the  soft  skills  area  as  has  been  the  case. 
This  observation  Is  supported  by  reference  to  Paragraph  A-2,  which  deals  with 
job  analysis  information  sources  suggested  as  a  basis  for  developing  a  task 
Inventory.  Of  15  possible  sources  mentioned  in  this  paragraph,  only  one  repre¬ 
sents  new  Information  from  job  incumbents  obtained  specifically  for  the  purpose 
of  developing  the  task  Inventory. 

A  second  observation,  perhaps  controversial,  is  that  it  appears  to  the  present 
writer  that  both  of  the  references  discussed  above  Implicitly  assume  that  a 
"true  expert"  Is  to  be  found  somewhere.  The  Idea  is  that  the  "true  expert," 
once  located,  will  be  able  to  specify  his  Inputs  and  outputs.  Finally,  one 
might  conclude  that  there  may  also  be  an  assumption  that  there  Is  one  correct 
way  to  do  the  "soft  skills"  job.  In  the  remainder  of  this  paper,  all  of  these 
possible  assumptions  will  be  questioned. 

WHO  IS  THE  TRUE  EXPERT? 

Without  desiring  to  be  critical  of  an  Interesting  and  innovative  beginning, 
it  is  Instructive  to  search  momentarily  for  a  true  expert.  Whitmore  (1972), 
in  discussing  his  concept  of  the  job  model,  notes  that  "The  Identification 
and  definition  of  job  functions  from  system  characteristics  and  the  formula¬ 
tion  of  the  behavioral  processes  underlying  each  function  requires  behavioral 
science  expertise."  This  suggests  the  possibility  that  the  behavioral 
scientist  may  be  "the  true  expert,"  at  least  In  this  complex  soft  skill  area. 
The  validity  of  this  assumption  may  be  Illustrated  by  a  quick  comparison  ol 
T-group  protagonists  and  antagonists. 


IV- 37 


T-group  (sensitivity)  training  Is  a  fairly  widely  used  form  of  training  to 
which  executives  and  leaders  frequently  are  exposed  In  the  belief  that  It 
will  "develop"  them.  Bradford  (1972),  as  one  protagonist,  describes  how 
sensitivity  training  works.  He  concludes  that,  while  not  all  Individuals 
prc  It  equally  from  such  training,  moat  benefit  to  some  extent  and  "... 
the  majority  of  participants  have  learned  and  grown." 

Adair  (1968)  criticizes  sensitivity  training  on  four  counts: 

a.  Such  groups  have  no  formal  tasks,  which  means  that  they  do  not  have 
task  and  team  maintenance  responsibilities  as  real  groups  In  formal  organiza¬ 
tions  do.  Thus,  the  experience  Is  dissimilar  to  the  on-the-job  conditions  of 
performance,  which  should  decrease  transfer  of  training. 

b.  Several  assumptions  underlying  this  approach  probably  are  Inaccurate. 

An  example  Is  the  assumption  that  leadership  functions  should  be  distributed 
equally  among  the  members  of  the  group.  Adair  suggests  the  possibility  that 
this  may  well  not  be  true  when  there  are  real  tasks  which  must  be  achieved 
under  difficult  situations. 

c.  This  approach  as  a  universal  method  of  leadership  training  Is  culture- 
bound.  (It  should  be  noted  that  this  criticism  Is  not  valid.  If  transfer  of 
training  Is  planned  only  to  the  culture  to  which  It  Is  "bound.") 

d.  There  Is  a  tendency  for  group  psychotherapy  to  invade  the  training 
scene.  With  this  Invasion  may  come  "preconceptions"  which  may  not  be  appro¬ 
priate  for  leaders  of'normal"  people. 

The  above  attempt  to  pit  two  schools  of  thought  against  one  another  Is,  ad¬ 
mittedly,  somewhat  unfair.  Clearly,  there  would  be  points  of  agreement. 
However,  the  major  point  Is  that  there  clearly  are  differences  In  objectives. 
Implied  or  explicit.  That  Is,  protagonists  would  Identify  one  set  of  ob¬ 
jectives  for  leader  training  and  antagonists  another  set.  The  differences 
In  objectives  produce  differences  In  the  training  approach  taken.  Further, 
the  criteria  which  would  be  used  to  evaluate  the  effectiveness  of  the  training 
would  be  quite  substantially  different. 

Clearly,  the  "true  expert"  may  or  may  not  be  the  behavioral  scientist.  To 
the  extent  that  he  has  theoretical  biases,  his  degree  of  "expertness"  may 
be  compromised.  And  I  believe  that  no  one  would  assert  that  behavioral 
sciences  are  lacking  In  theoretical  biases. 

There  Is  yet  another  reason  why  the  behavioral  scientist  may  not  be  the 
"true  expert."  Most  behavioral  scientists,  as  McGregor  (1960),  points  out, 
have  never  been  required  to  perform  In  the  role  of  a  line  executive.  McGregor, 
without  such  experience,  had  believed  that  a  leader  could  operate  essentially 
as  an  organizational  advisor,  and  tnat  good  human  relations  would  be  quite 
effective  In  relieving  discord  and  disagreement.  Experience  demonstrated  to 
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him  that  a  leader  cannot  avoid  the  exercise  of  authority*  The  leader  can¬ 
not  delegate  the  responsibility  which  he  alone  must  assume  for  the  workings 
of  his  organization. 

This  point  has  been  made  with  trepidation,  and  with  the  anticipation  of 
challenge.  It  Is  in  fact  correct  that  experience  In  the  role  is  not  neces¬ 
sary  In  order  to  be  able  to  describe  the  role  through  rigorous  observational 
techniques.  So  some  behavioral  scientists  may  be  able  to  rise  beyond  in¬ 
experience  to  become  "true  experts."  However,  it  is  very  difficult  to  ob¬ 
serve  intangibles.  Leadership  and  executive  skills  are  intangible.  The 
necessary  recourse,  therefore,  is  to  study  experimentally  the  impact  of 
variation  in  role  behaviors  on  organizational  outcomes.  It  should  be  recog¬ 
nized  that  this  Is  an  extremely  difficult  proposition.  Several  questions 
Immediately  are  raised,  which  bear  not  only  on  systems  engineering  but  also 
on  evaluation: 

a.  What  should  be  used  as  the  criterion?  Should  it  be  evaluations 

by  superiors?  Should  It  be  evaluations  by  subordinates?  Should  it  be  soma 
rating  or  objective  measure  of  organizational  performance? 

b.  What  organization  can  be  made  available  for  experimental  study? 

Real  organizations  exist  in  a  world  that  usually  is  competitive.  Only  rarely 
are  real  organizations  available  for  experimental  study.  (And  they  also  are 
hard  to  simulate.) 

c.  How  much  time  can  be  devoted  to  experimental  study?  The  time  required 
varies,  depending  on  the  object  of  the  study.  In  a  cross-sectional  study, 
relatively  little  time  Is  required.  In  a  longitudinal  study,  months  or  per¬ 
haps  even  years  may  be  required. 

WHAT  IS  THE  ACTUAL  NATURE  OF  THE  REAL  WORLD? 

Bruner  (1956)  discusses  two  different  kinds  of  concepts.  The  first,  con¬ 
junctive  concepts,  consists  of  those  in  which  the  logic  is  essentially  trans¬ 
itive.  That  is,  If  in  Middletown  only  residents  can  vote.  It  is  possible  to 
Infer  by  knowing  that  an  individual  has  voted  that  he  Is  a  resident.  Man- 
machine  systems  basically  are  conjunctive  systems.  The  logic  of  man-machine 
Interaction  Is  a  transitive  logic. 

The  second  type  of  concept  is  disjunctive,  in  which  this  kind  of  transitivity 
does  not  occur.  For  exanqple,  if  in  Middletown  either  residents,  taxpayers, 
or  persons  who  work  within  the  city  may  vote,  then  it  is  not  possible  to  infer 
the  individual's  category  from  knowing  that  he  votes.  It  is  very  likely  that 
most  man  ascendant  systems  are  characterized  by  disjunctive  logic,  where 
several  different  solutions  may  work  in  any  given  problem  situation.  (This 
is  recognized  In  the  Idiom  by  the  statement  that  any  solution,  well  implemented. 
Is  better  than  no  solution.) 
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Of  course,  It  is  possible  that  organizational  leadership  is  not  actually 
disjunctive.  It  may  only  appear  so  because  of  the  many  variables  ^ich 
may  affect  the  impact  of  any  given  leadership  behavior  or  approach. 

Further,  there  may  be  moderator  variables  which  actually  change  the  rela- 
ti  ciehip  between  a  given  leadership  behavior  and  the  expected  outcome. 
(Fiedler’s  contingency  model  illustrates  the  use  of  moderator  variables 
in  leadership  theory  building.)  Several  studies  illustrate  some  of  these 
variables. 

a.  Organizational  Climate.  In  a  relatively  well-known  study, 

Fleishman  (1953)  reported  the  effects  Of  leadership  training  for  foremen 
after  the  foreman  had  returned  to  their  Industrial  situation.  The  leader¬ 
ship  training  itself  consisted  of  a  focus  on  Initiating  structure  and  shov¬ 
ing  consideration  dimensions.  Fleishman  reported  that  the  effects  of  the 
training  appeared  rather  small  when  a  later  follow-up  was  made.  The  kind 
of  supervisor  (leadership  climate)  under  which  the  foreman  worked  was  more 
related  to  his  own  behavior  than  either  training  or  Ic^k  of  it.  Foremen 
working  under  a  considerate  supervisor  tended  to  express  more  considerate 
attitudes  toward  their  workers,  and  received  consideration  scores  from 
their  workers  which  were  higher. 

b.  Organizational  Structure.  Forehand  and  Gilmer  (1964)  further 
analyzed  the  effect  of  climate  on  leader  behavior.  They  noted  that  organi¬ 
zational  climate  may  be  determined  not  only  by  the  leadership  pattern  of 
one's  own  supervisors,  but  also  by  size,  organizational  structure,  systems 
complexity,  and  goals.  Presumably,  any  of  these  factors  could  negate  or 
enhance  the  impact  of  leadership  training,  or  modify  the  relationship  between 
given  leadership  behavidr  and  system  outcomes. 

c.  Nonlinear  Relationships.  A  report  of  a  seminar  held  by  the  Founda¬ 
tion  for  Research  on  Human  Behavior  (1955)  describes  several  of  these  non¬ 
linear  relationships: 

(1)  Production  Pressure.  When  production  pressure  was  low, 

an  increase  in  pressure  increased  both  satisfaction  with  supervisor  and 
production,  to  a  point,  beyond  which  satisfaction  dropped  sharply. 

(2)  Group  Cohesiveness.  Group  Cohesion  is  positively  correlated 
with  productivity  when  the  company  is  perceived  as  supportive,  but  negatively 
correlated  with  productivity  when  the  company  is  perceived  as  threatening. 
(This  illustrates  a  moderator  variable.) 

(3)  Closeness  of  Supervision.  Production  can  be  high  either  way, 
but  morale  suffers  with  excessively  close  supervision,  and  a  large  invest¬ 
ment  of  supervisor  energy  must  be  expended  also. 

d.  Subordinate  Expectations.  Jasinskl  (1959)  reported  an  interesting 
study  from  the  automobile  industry,  in  which  workers  and  foremen  alike  had 
proper  and  Idealized  descriptions  of  what  the  supervisor’s  role  ought  to  be. 
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However,  In  actual  practice  the  foreman's  behavior  wae  very  different  from 
the  verbal  description  of  the  ideal  role.  Further,  Jasinski  found  that 
workers  really  expected  a  different  type  of  behavior  from  the  foreman  than 
they  said  they  wanted,  and  the  work  environment  precluded  the  lengthy  inter¬ 
action  between  workers  and  foremen  that  would  have  been  required  by  "desired" 
behavior.  Finally,  the  supervisors  of  the  foremen  gave  little  weight  to 
foreman  relations  with  men  in  their  evaluations  of  the  foremen. 

When  leadership  Is  taught  and  evaluated  in  a  service  school,  the  service 
school  essentially  is  serving  a  customer.  The  customer  is  the  commander  in 
the  field  who  uses  the  product  generated  by  the  school.  It  clearly  is  im¬ 
portant  to  develop  and  evaluate  the  right  thine.  One  further  example  of  the 
cataclysmic  results  that  may  occur  when  the  wrong  thing  is  developed  is  found 
in  a  report  by  Sikes  (1962)  of  a  course  on  human  relations.  This  course  was 
given  to  all  managers  and  supervisors  in  a  medium  sized  contracting  firm. 

The  evaluation  tool  was  a  questionnaire  to  supervisors.  One  question  was, 

"Was  the  course  as  a  whole  a  success  or  a  failure"?  The  summary  of  answers 
was: 

Success  0 

Failure  83 

No  Opinion  14 

These  startling  findings  occurred  because  participants  felt  the  course  had 
not  made  any  lasting  Improvement  in  the  attitudes  of  senior  management. 
However,  the  cataclysmic  further  result  was  a  crisis  in  the  conflict  between 
expectations  held  by  supervisors  and  by  senior  management  concerning  the  role 
of  management.  This  conflict  led  to  resignations  of  19  of  the  97  super¬ 
visors  who  had  attended  the  course.  (In  the  two  years  preceding  the  course, 
only  2  had  left.)  Further,  25  more  were  known  to  have  applied  for  other 
jobs. 

HOW  TO  MEASURE  THE  UNMEASURABLE? 

At  the  outset,  it  was  noted  that  ultimate  criteria  are  rarely  available  for 
measuring  the  impact  of  leadership  training  and  development.  There  are 
probably  two  reasons.  The  first  is  the  difficulty  of  obtaining  objective 
criterion  measures  in  real  organizations.  Realistically,  this  does  present 
problems. 

Another  difficulty  is  that  it  sometimes  is  embarrassing  to  the  researcher 
to  be  able  to  obtain  such  criteria,  when  they  are  available. 

Korman  (1966)  distinguishes  two  kinds  of  validity,  concurrent  and  predictive. 
Concurrent  validity  is  derived  from  measurements  taken  at  the  same  time  that 
measurements  are  made  of  the  so-called  leader  manipulation.  Predictive  valid¬ 
ity  is  derived  from  a  longitudinal  :^tudy,  in  which  a  leadership  manipulation 
is  made,  and  efforts  are  then  made  ::o  determine  whether  Individuals  in  the 
experimental  group  behave  in  different  (hopefully  better)  ways  on  the  job. 
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The  embarrassing  part  about  such  criteria  is  that  they  frequently  lead 
to  the  conclusion  that  no  change  has  occurred »  especially  where  the  criteria 
consist  of  objective  measures  of  organisational  performance*  It  will  be 
recalled  that  Fleishman's  finding  of  no  impact  came  from  a  longitudinal 
study*  In  his  own  review  of  the  literature,  Korman  found  quite  similar 
results*  His  focus  was  on  initiating  structure  and  shoving  consideration 
as  leadership  behavior  variables*  Host  of  the  studies  reported  concurrent 
validity,  which  Korman  notes  does  not  give  good  evidence  of  causality* 

Further,  the  objective  criteria,  when  they  were  used,  yielded  extremely  low 
correlations  with  either  consideration  or  Initiating  structure*  This  is  a 
finding  that  is,  of  course,  repeated  fairly  frequently  in  other  studies* 

Korman 's  other  results  are  interesting  in  their  own  right.  Table  1  on 
the  following  page  summarises  his  review.  It  shows  correlations  between 
initiating  structure  and  shoving  consideration  variables,  and  a  variety  of 
other  measures.  In  Table  1,  concurrent  and  predictive  validity  data  have 
been  mixed.  As  can  be  seen,  there  is  substantial  variability  in  the  findings, 
regardless  of  what  criterion  Is  used*  The  only  consistent  findings  appear 
to  be  correlations  between  subordinate  ratings  of  leaders  and  shoving  con¬ 
sideration  behavior.  Korman  concludes  that  his  survey  of  the  literature 
does  not  provide  any  reason  for  saying  that  these  leader  behavior  variables 
can  predict  work  group  performance,  or  under  what  conditions  they  might  af¬ 
fect  work  group  perf  nuance,  a  somewhat  dismal  picture. 

These  findings  repres  nt  only  a  marginal  sampling  from  the  available  liter¬ 
ature*  However,  they  do  nonetheless  illustrate  the  extreme  difficulty  of 
determining  what  should  be  taught  as  leadership,  how  it  should  be  taught, 
and  how  the  effectiveness  of  the  resulting  skill  should  be  evaluated* 

IF  NOT  ULTIMATE  CRITERIA,  THEN  WHAT? 

While  so-called  ultimate  criteria  may  be  available  within  organizational 
settings,  they  generally  are  not  available  to  the  trainer.  This  has  led 
to  a  proliferation  of  intermediate  criteria  ^ich  have  been  used  for  assess¬ 
ment  purposes.  Some  of  the  vehicles  for  leadership  assessment  have  been: 

Situational  Performance  Tests 

Situational  Paper-and-Pencil  Tests 

Personality  Tests 

Peer  Ratings 

Ratings  by  Seniors 

Leadership  Grades 

This  list  is  by  no  means  inclusive*  It  is  intended  primarily  to  illustrate 
the  diversity  of  approaches  in  current  use. 


Table  1 


Correlations  Between  Various  Criteria  and 
Initiating  Structure  and  Showing  Consideration 


With 

Objective  Measures 
Superior  Ratings 
Peer  Ratings 
Self  Ratings 
Subordinate  Ratings 


Initiating  Structure 


N 

Range 

S 

-.05  to  .23 

8 

-.39  to  .13 

8 

-.11  to  .14 

3 

-.41  to  .45 

3 

.05  to  .22 

Showing  Consideration 
N  Range 


5 

-.11 

to 

.14 

8 

-.06 

to 

.29 

8 

-.17 

to 

.03 

3 

-.02 

to 

-.46 

3 

.24 

to 

.51 
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situational  Tests*  Theae  tests  generally  depend  for  predictive  validity 
on  the  similarity  between  the  performance  requirement  (Including  group  leader 
ship  and  direction)  of  the  test  situation  and  that  of  the  ultimate  perform¬ 
ance  situation.  However,  situational  tests  have  been  plagued  by  many  common 
psychometric  problems.  Among  these  are  the  following: 

.  The  difficulty  of  establishing  parallel  situations, 

so  that  different  leaders  will  be  assessed  under  comparable 
conditions  of  performance. 

•  The  difficulty  of  establishing  situations  which  in  fact 
present  real  life  performance  requirements  at  other  than 
a  face  valid  level. 

.  The  cost  In  terms  of  time  and  administrative  difficulty  In 
running  good  situational  tests. 

.  The  difficulty  of  training  assessors  so  that  they  respond  to 
the  same  leader  behaviors  In  the  same  way,  or  even  observe 
the  same  leader  behavior  variables. 

.  The  difficulty  of  providing  subordinates  who  respond  as  sub¬ 
ordinates  would  In  real  life.  (In  the  test  situation  they 
hardly  are  subject  to  quite  the  same  actual  leader  power  as 
In  a  real  unit.) 

Situational  Paper-and-Pencll  Tests.  An  example  is  the  In-Basket  Test. 

A  problem  with  this  kind  of  test  Is  that  one  of  the  most  critical  dimensions 
of  leadership  skill  probably  is  Interpersonal  competence.  The  austere  con¬ 
text  of  an  In-Basket  Test  Is  hardly  appropriate  for  assessing  this  kind  of 
variable.  Further,  in  the  military  setting,  there  may  be  some  degree  of 
stress  associated  with  the  performance  requirement,  which  Is  difficult  to 
simulate  In  an  in-basket  or  other  similar  test.  Finally,  it  Is  Impossible 
to  assess  group  maintenance  or  Individual  support  leader  behavior  skills 
in  a  “solo"  performance. 

Personality  Tests.  Typical  of  this  approach  is  the  self-assessment 
test,  which  commonly  takes  the  form  of  measuring  the  Individual’s  proclivi¬ 
ties  toward  authoritarian,  democratic,  or  laissez-faire  leadership.  The 
fallacies  of  this  approach  are  obvious. 

Peer  Ratings.  While  It  Is  difficult  to  know  exactly  what  peer  ratings 
measure,  they  generally  have  had  better  predictive  validity  than  almost 
any  other  single  approach.  This  may  be  because  they  probably  measure  the 
dimension  of  interpersonal  competence  and  the  ability  to  work  within  a  co¬ 
ordinated  group  better  than  do  most  other  approaches.  However,  even  so, 
peer  ratings  rarely  account  for  more  than  25%  of  the  total  variance  In 
subsequent  leadership  performance,  and  are  always  subject  to  the  vagaries 


of  the  peeve  who  are  completing  the  a.  That  Is »  If  a  group  decides  to  do 
80,  "cooperate  and  graduate"  can  be  made  to  work  extremely  well,  nonethe¬ 
less,  they  are  a  "method  of  choice,"  especially  when  used  In  conjunction 
%d.th  other  measures. 

Ratings  by  Seniors*  It  is  very  difficult  to  assess  the  extent  to  which 
ratings  by  seniors  are  predictive  in  an  independent  sense.  That  is,  in 
a  formal  school  setting,  peer  ratings  and  tactical  officer  ratings  are 
found  to  be  fairly  highly  correlated.  Ratings  by  tactical  officers  are 
generally  less  reliable  than  ratings  by  peers. 

Leadership  Grades.  Finally,  leadership  grades  have  been  developed 
from  paper-and-pencil  knowledge  tescs  following  leadership  Instruction. 

In  unpublished  research,  HumRRO  Division  No.  4  has  found  them  on  occasion 
even  to  be  negatively  correlated  with  leadership  grades  obtained  through 
a  combination  of  peer  ratings  and  tactical  officer  ratings.  This  suggests 
that  they  probably  would  not  predict  subsequent  performance,  in  that  peer 
ratings  generally  do. 

A  DISJUNCTIVE  APPROACH  FOR  A  DISJUNCTIVE  CONCEPT 

The  picture  to  this  point  has  been  negative,  and  unfairly  so.  It  was  pre¬ 
sented  in  this  fashion  to  illustrate  the  difficulties  in  both  soft  skills 
systems  engineering,  and  in  the  evaluation  of  soft  skills.  Soft  skills 
systems  engineering  is  extraordinarily  difficult  because  of  the  absence 
of  a  "true  expert,"  and  because  of  the  intangible  nature  of  these  skills. 
Because  the  evaluation  process  is  necessarily  dependent  on  Identification 
of  what  Is  to  be  evaluated,  the  development  of  soft  skill  evaluation  pro¬ 
cedures  has  necessarily  been  Impeded  by  the  same  problems. 

However,  It  may  be  appropriate  to  make  an  assertion  at  this  point.  While 
It  Is  extremely  unlikely  that  any  massive  breakthrough  Is  ever  going  to 
occur  In  soft  skills  systems  engineering,  approaches  such  as  the  behavioral 
model  are  conceptually  sound,  and  will  eventually  produce  a  sound  picture 
of  what  it  Is  that  needs  to  be  trained.  By  the  same  token,  it  will  then 
become  feasible  to  develop  evaluation  techniques.  The  requirement  is  simply 
for  patience  and  continued  effort,  with  the  realization  that  this  is  simply 
a  much  more  difflctilt  problem  than  hard  skills  systems  engineering. 

As  an  example  of  possible  future  evaluation  approaches.  It  Is  appropriate 
to  examine  the  "scientific  assessment  center."  Management  assessment  centers 
are  becoming  more  frequent  In  industry,  probably  because,  as  Byham  (1972) 
asserts,  they  have  shown  themselves  "...to  be  a  more  reliable  Indicator 
of  future  success  than  any  other  tool  yet  devised." 

The  assessment  center  works,  probably,  because  It  Is  so  comprehensive  In 
what  It  measures*  (Critics  might  argue  that  If  enough  of  an  Individual's 
behavior  Is  observed,  probably  It  W(  uld  be  possible  to  make  predictions  to 
any  future  activity.)  Byham  Identifies  25  "common  dimensions"  of  managerial 
success.  They  are  as  follows: 


1.  Impact 

2.  Energy 

3.  Oral  communication  skill 

4.  Oral  presentation  skill 

5.  Written  communication  skill 

6.  Creativity 

7.  Range  of  Interest 

8.  Stress  tolerance 

9.  Motivation 

10.  Work  standards 

11.  Leadership 

12 .  Salesmanship 

13.  Sensitivity 


14.  Listening  skill 

15.  Flexibility 

16.  Tenacity 

17.  Risk  taking 

18 .  Initiative 

19 .  Independence 

20.  Planning  and  Organising 

21 .  Management  control 

22.  Use  of  delegation 

23.  Problem  analysis 

24 .  Judgment 

25.  Decisiveness 


Byham  also  Identifies  eight  techniques  vdilch  nay  be  used  In  an  assessment 
center  for  evaluating  Individuals  on  these  dimensions.  They  are: 

Interview 

Management  games 

In-Basket  and  Interview 

Leaderless  Group  Discussion  (Assigned) 

Leaderless  Group  Discussion  (Non-Asslgned) 

Fact  Finding  and  Decision  Making 
Analysis  Presentation 
Interview  Simulation 

Some  of  these  may  be  better  for  assessing  one  or  more  of  the  preceding  dimen¬ 
sions  than  others.  In  the  assessment  center,  the  optimum  measurement  approach 
Is  selected  for  each  measurement  to  be  obtained. 

According  to  Byhsm,  psychologists  are  considerably  superior  to  untrained 
line  assessors  In the  assessment  center  context.  However,  they  are  not  always 
available,  so  companies  tend  to  establish  pools  of  trained  assessors.  With 
either,  the  predictive  validity  of  the  assessment  center  Is  quite  good. 

Bray,  Grant,  and  Campbell  (1972)  make  an  essentially  similar  point. 

The  difficulty  with  assessment  centers  is,  of  course,  that  they  are  ex¬ 
pensive  to  operate,  and  have  a  low  throughput  of  assessed  potential  leaders. 
Where  management  turnover  Is  low,  this  Is  no  problem.  In  the  Army,  with 
high  Input  of  junior  leaders.  It  might  be  a  problem. 

However,  In  the  context  of  the  present  paper,  the  assessment  center  concept 
Is  probably  worth  examining  as  the  prototype  for  soft  skills  evaluation 
procedures  development .  The  assessment  center  "covers  the  waterfront." 
Similarly,  It  Is  probable  that  the  evaluation  of  any  soft  skill  will  neces¬ 
sarily  cover  a  multitude  of  dimensions.  Single  unitary  measures  probably 
will  not  suffice.  (This  suggests  that  leadership  training  also  will  need 
to  cover  a  variety  of  dimensions.)  Finally,  It  la  likely  thkt  leadership 
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evaluation  procedures  will  graduall r  Improve  with  experience  and  research, 
just  as  leadership  development  and  i:ralnlng  procedures  Improve  through  the 
same  means*  In  each  case,  continues  effort  should  provide  Increasingly 
Improved  payoffs. 

As  a  final  example  of  the  Inextrlcaole  Intertwining  of  leadership  research 
and  leadership  development.  It  Is  appropriate  to  examine  a  probable  new 
direction  In  the  assessment  of  organizational  leadership.  Students  of  small 
group  theory  will  affirm  that  substantial  progress  In  understanding  of  small 
group  dynamics  came  only  after  researchers  turned  to  the  study  of  processes 
that  occur  in  small  groups.  Examination  of  the  assessment  dimensions  just 
cited  shows  that  they  are  essentially  person  oriented.  Thus,  they  are 
likely  to  be  limited  in  their  predictive  power. 

However,  a  few  researchers  have  been  turning  to  a  study  of  organizational 
processes  In  order  to  understand  organizational  leadership  more  fully.  This 
generally  requires  some  sort  of  simulation  of  an  organization,  because  real 
organizations  cannot  be  "replayed.**  The  capacity  to  repeat  a  standardized 
experience  is,  of  course,  necessary  In  order  to  develop  a  basis  for  Infer¬ 
ential  testing.  As  an  example  of  this  work,  Olmstead  (1972)  has  developed 
a  simulation  of  a  battalion  engaged  in  a  counterinsurgency  operation.  The 
simulation  is  designed  to  permit  measurement  of  organizational  process  vari¬ 
ables  identified  by  Schein  (1965)  as  the  Adaptive  Coping  Cycle.  These  vari¬ 
ables,  and  their  correlations  with  Independent  measures  of  the  adequacy  of 
mission  accomplishment,  are: 


Sensing  a  change  in  the  internal  or  external  environment  .92 
Communicating  the  information  to  a  decision  maker  .83 
Making  a  decision  with  the  Information  .70 
Stabilizing  internal  changes  .11 
Communicating  change  decisions  .71 
Coping  actions  .72 
Obtaining  feedback  as  to  decision  effectiveness  .03 


It  is  beyond  the  scope  of  the  present  paper  to  present  Olmstead *s  findings 
In  greater  depth.  The  significant  point  is  that  organizational  leadership 
possibly  should  be  viewed  as  the  capacity  to  cause  certain  crucially  im¬ 
portant  functions  to  be  performed  well.  This  would  have  equally  significant 
implications  for  assessu^ent  of  leadership.  The  question  would  be,  "Leader¬ 
ship  for  what?"  And  the  assessment  technique  would  be  examination  of  the 
leader *s  capacity  for  making  processes  work  well. 

As  a  contrast,  Olmstead  also  obtained  many  of  the  conventional  paper-and- 
pencil  measures  often  used  to  assess  leaders  themselves,  with  miniscule 
results.  In  this  simulation,  the  sj-called  "personality**  measures  used  did 
not  tap  the  capacity  to  make  the  process  go  smoothly.  Also,  they  did  not 
tap  organizational  effectiveness.  The  implications  of  these  findings  for 
assessment  of  leadership  skills  are  profound. 
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SUMMARY 


In  this  paper,  It  has  been  asserted  that  the  development  of  leadership 
evaluation  procedures  has  been  impeded  by  the  seme  obstacles  that  have 
slo.ed  the  development  of  soft  skills  systems  engineering  techniques. 

It  is  difficult  to  find  a  "true  expert,"  and  the  lacuiid>ent  Is  frequently 
unable  to  verbalise  what  he  actually  does  because  what  he  does  Is  Intang¬ 
ible.  This  requires  that  knowledge  be  developed  throu^  the  arduous 
method  of  experimental  study  of  leadership  processes  by  researchers  and 
other  observers.  However,  the  general  unavailability  of  ultimate  criteria 
of  leadership  effectiveness  makes  this  a  aather  slower  process  than  It 
otherwise  might  be. 

The  dearth  of  ultimate  criteria  has  led  to  a  proliferation  of  Intermediate 
criteria.  The  researcher's  own  bias  has  often  influenced  the  Intermediate 
criteria  he  has  chosen  to  study,  and  the  hypotheses  he  has  chosen  to  test. 
Researcher  bias  has  also  Influenced  the  types  of  leadership  processes  he 
has  chosen  to  study. 

Despite  the  handicaps  that  shackle  the  researcher,  among  which  Is  the  fact 
that  he  rarely  has  been  in  the  position  of  a  line  executive  and  thus  does 
not  know  how  It  feels,  the  researcher  probably  Is  Indlspenslble  in  the  soft 
skills  area  as  Whitmore  has  noted.  This  Is  not  because  of  what  he  knows 
substantively.  Indeed,  he  may  be  more  dangerous  because  of  his  substantive 
knowledge.  Rather,  the  researcher's  value  stems  from  his  training  In  scien¬ 
tific  methodology  and  from  the  fact  that  he  Is  by  virtue  of  that  training 
a  highly  qualified  observer.  This  assumes,  of  course,  that  his  own  theoreti¬ 
cal  biases  do  not  blind  him. 

It  Is  probable  that  most  soft  skill  areas  are  disjunctive  In  nature.  This 
creates  substantial  difficulties  for  anyone  attempting  to  do  systems  engineer¬ 
ing  work.  However,  substantial  progress  has  been  made  In  recent  years  In 
the  development  of  knowledge  about  leadership,  as  an  exsisple  of  one  soft 
skill.  The  assessment  center  concept  was  discussed  briefly  as  a  possible 
protot3rps  of  future  evaluation  approaches. 

Though  not  Identified  specifically  In  the  paper,  there  are  needs  that  are 
worth  citing: 

a.  Continued  research  to  learn  what  the  substantive  nature  of  the 
leadership  phenomenon  Is.  Such  research  will  continue  to  produce  progress 
in  the  development  of  leadership  training,  and  will  permit  parallel  progress 
in  the  development  of  evaluation  techniques. 

b.  Continued  search  for  Intermediate  criteria.  This  will  require  the 
availability  of  organizations  for  study,  perhaps  through  simulation  tech¬ 
niques  such  as  those  Olmstead  (1972)  has  used.  Further,  there  is  a  need 
for  enlightened  decision  makers  who  will  tolerate  other  than  face  valid 
measures  for  leadership  assesssient. 

c.  Continued  work  to  learn  how  to  modify  an  Individual  to  be  more 
effective  as  a  leader. 


« 
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TRAINING  QUALITY  CONTROL 


Evaluation  of  Leadership  and  Coimaunlcatlve  Skills 
James  L.  Sherrill* 

U.  S.  Amy  Adjuteint  General  School 

Testing  and  evaluation  Is  a  vital  part  of  training  quality  control  and 
systems  engineering  of  training.  As  we  continue  our  applications  of 
systems  engineering  we  are  becoming  increasingly  aware  of  real  problems 
dealing  with  those  content  areas  Identified  as  "soft  skills".  VThether 
we  are  capable  of  dealing  with  the  soft  skill  areas  in  a  systems  engine¬ 
ered  fashion  is  questionable. 

A  Few  General  Quotes  .  .  . 

Meckenburger  and  Wilson  (1971);  "Testing,  an  issue  educators  have  slipped 
under  the  mg  of  collective  guilt  for  two  decades  ...  (p.  UlO)." 

Glasser  (1969)1  cited  Bowers  recent  two-year  investigation  of  students  in 
99  colleges  emd  universities  which  found  that  at  least  55  %  of  the  students 
cheated  on  tests.  Bower  also  reported  finding  a  significant  number  of  in¬ 
stances  in  which  students  "negatively  cheated":  that  is,  they  would  delib¬ 
erately  misinfom  their  fellow  students  In  the  expectation  that  the  student's 
standing  would  be  lowered  and  their  own  thereby  raised. 

Carl  Rogers  (1969):  "While  it  is  clear  that  examination  -  passing  ability  is 
a  useful  skill,  cuid  has  a  place  in  professional  training,  it  almost  certainly 
emphasizes  rote  learning  and  mental  agility  rather  than  originality  of  thought 
and  scientific  curiosity,  traits  which  in  the  long  r\m  sure  more  valuable 
(p.  173)."  Later  Dr.  Rogers  observed:  "It  is  difficult  to  exaggerate  the 
damage  done  a  promising  graduate  student  by  the  completely  faliclous  assxunp- 
tion  that  they  learn  by  being  threatened,  time  after  time,  with  catastrophic 
failure  (p.  177).” 

Ccorver  1972;  "Some  day  college  courses  on  testing  will  start  teaching  stu¬ 
dents  that  the  traditional  test-building  procedures  are  not  appropriate  for 
measuring  chcmge,  or  gain,  or  Improvement  but  that  they  are  appropriate  only 
for  measuring  individual  differences  (p.  30)." 


*The  views  esqpressed  in  this  paper  are  those  of  the  author  and  do  not 
necessco'lly  reflect  those  of  the  Adjutant  General  School  or  Department 
of  the  Army. 


Lessenger  (1971)?  "We  have  virtually  no  neasurenent  of  the  results  that 
our  enterprise  yields.  All  ...  Indices  ...  measure  our  skill  as  func¬ 
tional  managers;  not  a  single  one  evaluates  our  effectiveness  as  educa- 
tlo- al  managers  (p.  13)." 

A  Fev  Quotes  On  Aptitude  and  Personality  Measures  . . . 

Lee  J.  Cronbach  (1967):  during  a  discussion  on  measures  of  aptitudes, 
cited  McNemar  (l9^4)  as  noting  the  rather  small  success  of  differential 
aptitude  tests  in  predicting  which  academic  subjects  the  person  will  learn 
most  easily.  Cronbach  then  observed  "We  haven't  the  faintest  evidence,  for 
exanxple,  what  constitutes  mathematical  aptitude  ...  (p.  27)." 

Oscar  K.  Buros  (1970):  on  personality  assessment  instriiments;  "Despite  the 
tremenaous  amount  of  research  devoted  to  ...  widely  used  tests,  they  have 
not  been  replaced  by  instriiments  more  acceptable  to  the  profession.  Nor 
has  the  research  resulted  In  a  consensus  cuaong  psychologists  concerning  the 
validities  of  a  particular  test.  The  vast  literature  on  personality  testing 
has  failed  to  produce  a  body  of  knowledge  generally  acceptable  to  psycholo¬ 
gists.  In  fact,  all  personality  instruments  msy  be  described  as  controver¬ 
sial,  eEu:h  with  its  own  following  of  devotees  (p.  xxxv)." 

Quotes  On  Prediction  . . . 

Edling  (1968);  "  ...  existing  paper  and  pencil  tests  ...  rarely  account 

for  more  than  half  of  the  variance  in  a  criterion  involving  complex  behavior 
(p.  l88)."  After  making  this  observation,  Edling  cited  Cronbach,  Mitzel, 
and  Buros  as  having  documented  this  assertion. 

Bond  and  Rlgney  (1970):  "When  well  controlled  transfer  studies  sire  attaapted, 
they  sure  often  negative;  a  host  of  studies  shows  that  school  or  college 
achievement,  for  example,  is  not  very  predictive  of  outside  school  achieve¬ 
ment  (p.  10)." 

Doll ,  Gunderson ,  and  Ryman  ( I969 ) :  "Prediction  of  personnel  performance  in 
indistrial  or  military  settings  tends  to  be  difficult  even  within  stable 
and  well-defined  orgcuiizational  structures.  Whether  actuarial  or  clinical 
methods  are  employed,  predictive  veiliditles  tend  to  be  low,  and  cross-va¬ 
lidities  often  are  insignificant." 

Holt  (196$);  from  a  review  of  forty-six  major  stiidies  concerned  with  re¬ 
lationships  between  college  performance  and  later  measures  of  adult  accom¬ 
plishment:"  ...  we  can  safely  conclude  that  college  grades  have  no  more  than 
a  very  modest  correlation  with  adult  success  no  matter  how  defined.  Refine¬ 
ments  in  experimental  methodology  are  extremely  unlikely  to  alter  this  gen¬ 
eralization  ...  (p.  1*5)." 
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Blum  and  Naylor  (1969);  Co-authors  of  a  text  in  wide  range  in  graduate 
level  industrial  psychology  courses,  €uid  strong  advocates  of  using  se¬ 
lective  and  predictive  measurements  in  industrial  work  forces;  ”The 
executive  category  is  a  most  difficult  group  for  which  to  develop  criteria 
•  •••  The  prohlems  tend  to  make  the  ..ask  of  objective  executive  perfornn 
ance  appraisal  an  exceedingly  coii5)lox  and  difficult  one  (p.  230).” 

Stake  (19T1)  comments;  ”Some  technical  errors  in  test  scores  are  small 
and  tolerable.  But  some  testing  errors  are  intolerably  large.  Today’s 
tests  can,  for  exaiDple,  measure  vocabulary  word- recognition  skills  with 
sufficient  accuracy.  They  can  not,  however,  adequately  measure  listening 
cozl^)rehension  or  the  ability  to  analyze  the  opposing  side  of  an  argument 
(p.  581*).” 

Jensen  (1970);  in  a  review  of  the  Thematic  Aperception  Test;  and  during 
a  general  discussion  of  the  scoring  reliability  of  projective  tests  com¬ 
mented:  ’’Scoring  reliability  below  .80  is  generally  considered  unaccept¬ 
able  in  scoring  essay  examinations  ...  (p.  932).” 

Ebell^Damrin  (i960):  leading  advocates  of  the  correlation  or  norm-referenced 
style  of  evaluation:  ”  ...  satisfactory  reader  reliability  for  composition 
type  tests,  both  metrk-remark  and  test-retest,  can  be  obtained  only  when  teams 
rather  than  individufiLLs  are  used.” 

Braddock  (1969);  ”  •••  it  is  remarkable  and  ironic  to  see  that  ...  almost 

no  researcher  has  discovered  any  more  correlation  between  knowledge  of  tra¬ 
ditional  grammar  and  quality  of  composition  than  would  be  fo\jnd,  for  example, 
between  grammar  and  geogrGq)hy.  (p.  4U8).” 

Discussion  and  Observations  ... 

The  preceding  quotations  raise  serious  questions  about  the  validity  of 
c\irrent  Anqy  Service  School  practices  in  the  evaluation  of  leadership  and 
communicative  skills  within  officer  professional  development  courses. 

lypical  of  our  present  practices  is  the.  Academic  Report,  DA  Form  1059*  Tables 
1  and  2  summarize  selected  portions  of  the  Academic  Reports  from  six  recent 
advanced  officer  eourse  classes  at  six  different  Amy  Service  Schools.  Ex¬ 
amination  of  these  data  show  considerable  differences  among  the  schools: 
differences  which  can  with  reetsonable  assurance  be  attributed  in  large  part 
to  individual  rater  differences  rather  than  differences  in  talentor  perform¬ 
ance  among  the  students. 


TABLE  1.  PERCENTAGE  OP  OFFICER  ADVANCED  COURSE  GRADUATES  RECOMMENDED  FOR 
FURTHER  SCHOOLING  AND  ASSIGNMENT  AS  AN  INSTRUCTOR  (Taken  from  the  aca¬ 
demic  reports  rendered  on  the  graduates  of  six  recent  adveuiced  officer 
classes  at  six  different  Service  Schools:  N's  ranged  from  38  to  173). 
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TABLE  2;  EVALUATIONS  OF  ORAL  EXPRESSION,  WRITING  ABILITY,  AND  CLASS  PAR¬ 
TICIPATION  RENDERED  ON  THE  GRADUAT’ES  OF  SIX  RECENT  ADVANCED  OFFICER 
CLASSES  AT  SIX  DIFFERENT  ARMY  SERVICE  SCHOOLS  (Taken  from  items  lU,  15, 
and  l6  of  the  Academic  Report,  N's  ranged  from  38  to  175). 
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•This  school  submitted  these  evEiluatlons  on  the  first  two  elements  only. 
Their  Career  Branch  returned  the  reports  directing  their  revision.  This 
resulted  in  their  adopting  the  policy  of:  the  ratings  rendered  on  the  grad¬ 
uates  of  each  advanced  class  on  items  lU,  15,  and  l6  will  be  distributed; 

20%  above  average.  60<  average,  euid  20  below  averase. 


Although  the  elements  summarized  in  Tables  1  and  2  are  typically  viewed 
as  assessments  of  what  a  student  learned  and  or  demonstrated  in  the 
course,  these  ratings  are  in  act\ial:.ty  attempts  at  prediction.  They  must 
be  so  categorized  because  they  are  relative  Judgments:  some  students 
possess  relatively  more  of  a  given  attribute  than  do  others. 

Even  were  all  the  ratings  in  the  Academic  Report  normalized  as  one  school 
has  done  (a  hi^ly  improbable  event)  it  is  quite  unlikely  that  these 
ratings  would  actually  be  predictive  of  any  reasonable  criteria  within 
the  operational  environment.  The  overwhelming  weight  of  evidence  from 
the  resecurch  literature  suggests  that  such  is  not  possible  aside  from 
questions  of  feasibility  and  practicality. 

Fiedler  (1967)  observed:  "Considering  the  fact  that  years  of  effort  have 
failed  to  produce  any  generally  valid  leadership  tests  • • •  it  seems  safe 
to  say  that  they  are  not  likely  to  appear  in  the  futvire  (p.  250).”  Later, 
Fiedler  commented;  ”  ...  we  can  offer  no  more  them  speciilation  to  ac¬ 
count  for  the  lack  of  positive  results  of  leadership  training  programs 
(p.  253.)”  Other  observers  have  drawn  the  same  conclusions  on  leadership 
training  as  well  as  training  in  communication  skills.  In  contrast  to  this 
widely  held  pessimistic  view  of  both  our  ability  to  predict  leadership 
success  and  the  effects  of  training  in  leadership  and  communication  skills. 
Dr.  Robert  F.  Neeli/,  Chairman,  Department  of  Psychology,  Indiana  University- 
Purdue  University  at  Indianapolis,  Indiana,  commented  that  several  major  re¬ 
search  efforts  conducted  in  industrial  settings  have  demonstrated  that  train¬ 
ing,  in  supervisory  skills  does  have  pay-off  value  in  terms  of  the  hard  cri¬ 
terion  of  worker  productivity.  One  unpublished  study  conducted  by  the  Uni¬ 
versity  of  Michigan  Institute  for  Social  Reseeurch  demonstrated  an  average 
thirteen  percent  increase  in  productivity  among  the  workers  whose  super¬ 
visors  had  been  given  specietl  training  in  supervision.  During  the  same 
time  period,  another  group  of  workers  whose  supervisors  did  not  receive  the 
training  averaged  only  a  three  percent  gain  in  productivity.  The  group  re¬ 
ceiving  the  training  emd  the  group  not  receiving  the  training  were  randomly 
assigned  from  matched  pairs  of  supervisors  equated  in  terms  of  type  of  work 
supervised  and  the  background  of  personnel  performing  the  work.  Dr.  Neel 
observed,  about  these  several  studies,  that  none  were  able  to  predict  dif¬ 
ferences  in  supervisory  ability  from  the  measures  taken  during  the  training. 
All  that  was  predictable  was  that  the  trained  group  would  outperform  the  un¬ 
trained  group.  Who  among  the  trained  group  would  excell  was  not  predictable. 

What  we  are  now  evaluating  about  leadership  and  communicative  skills  within 
our  officer  courses  and  the  accuracy  of  these  evaluations  is  highly  question¬ 
able.  If  the  reseeurch  literature  is  any  indication,  our  present  evaluation 
practices  are  meaningless. 


^  Personal  conversation  with  Dr.  Neel,  8  November,  1972. 


Assuming  that  our  evaluation  efforts  are  meaningless,  what  alternative 
approaches  appear  worthwhile?  An  additional  Msumption  underlying  this 
question  is  that  our  efforts  at  training  such  skills  will  not  suddenly 
cease:  we  have  too  much  faith  in  the  value  of  such  tred^ning. 

Similar  criticism  in  the  past  has  resulted  in  increased  effort  to  refine 
our  present  approaches:  rewriting  our  multiple  choice  questions  to  make 
them  better,  more  realistic,  less  ambiguous,  more  challenging  (discrimina¬ 
ting?  training  our  raters  to  be  more  xaniform,  to  produce  a  better 

spread  •••;  rewriting  o\xr  scoring  instructions,  refining  our  definitions, 
redesigning  the  check  list,  (the  newly  published  "Enlisted  Academic  Report,” 
CONARC  Form  736-R)^  revising  the  weights  ....  Our  response  is  almost  like 
a  knee  Jerk.  Every  time  we  travel  the  same  path:  with  about  the  same  results. 
Over  time  our  illusions  vanish  and  we  again  charge  in  the  same  direction. 

Let  Us  Try  a  Different  Approach  ••• 

In  the  following  are  offered  a  few  illustrative  examples  of  how  we  might 
proceed.  There  is  less  interest  in  pursuing  these  parti cxilar  exan5)le8  than 
in  suggesting  a  methodology  by  which  we  might  approach  identification  of 
relevant  skills  and  knowledges  related  to  leadership  and  communicative 
skills. 

Many  of  our  schools  do  involve  our  students  in  cleissroom  exercises  on  how 
to  write  staff  studies.  We  attenqpt  to  evaluate  this  skill.  Most  often, 
our  evaluations  of  these  writing  efforts  come  from  one  rater  and  are  xosually 
in  the  form  of  a  numerical  (70  to  100 )  or  letter  "grade”.  On  occasion  we 
use  two  or  more  raters  independently  and  average  their  evaluations.  The  ex¬ 
tent  of  agreement  among  raters  of  compositions  has  been  studied  extensively. 
Rarely  do  the  average  intercorrelations  among  a  group  of  expert  raters  aver¬ 
age  above  .50.  Even  assuming  Amy  Service  School  evaluations  of  composi¬ 
tions  approach  such  reliability  (and  our  scoring  reliability  is  most  probably 
consideredably  less),  is  this  good  enough?  In  essence,  this  means  that  at 
least  three  fourths  of  the  differences  we  see  in  students’  writing  ability 
come  from  the  instructors  who  rated  or  graded  the  papers  and  only  one  fourth 
of  the  differences  axe  related  to  the  students’  own  efforts.  And,  addition¬ 
ally  considering  the  large  number  of  man  hours  of  expensive  talent  invested 
in  grading  the  compositions,  why  don’t  we  stop?  Its  time  we  tried  something 
else. 

For  example.  Phrasing  the  "statement  of  the  problem  in  a  staff  study"is 
considered  of  major  inporteuxce.  Suppose  we  adopt  the  following  training 
objective: 
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Given  a  statenent  of  the  commanders  guidance  and  ten 
different  statements  (two  or  three  of  which  are  con¬ 
sidered  appropriate  end  the  remainder  containing  errors 
typically  made  in  phrasing  problem  statements ) ,  indicate 
those  statements  considered  appropriate  and  inappropriate. 


Another  training  objective  .... 

Given  a  problem  statement  and  thirty  statements  together 
with  source  (a  mixture  of  facts  and  assumptions),  identify 
those  which  are  facts,  those  which  are  assun^tions ,  and 
those  which  are  neither. 

On  the  preceding  two  objectives;  how  much  agreement  is  there  on  a  given 
statement?  How  do  eteknowledged  experts  differ  in  their  Judgments  vs 
naive  individuals?  Vftiich  statements  cannot  be  Judged  with  sufficient  con¬ 
sensus  by  acknowledged  experts?  What  recognition  skills  are  inherent  in 
making  such  Judgments?  These  questions  can  be  asked,  and  answered  by  an 
approach  which  attempts  to  issolate  on  one  variable  at  a  time.  How,  these 
sub-sets  of  the  entire  writing  task  are  obscured  and  contaminated  by  our 
efforts  to  make  over-all  Judgments  of  the  whole,  ^y  breaking  the  whole 
into  its  parts,  we  may  be  able  to  get  a  handle  on  some  of  the  real  skills 
required.  If  we  can  measure  them,  we  can  then  discover  how  to  teach  them. 

Now  we  do  not  know  what  we  are  measuring  or  teaching. 

One  may  respond  to  the  preceding  discussion:  "But  you  didn't  have  the 
student  write  a  problem  statement  or  a  fact  or  an  assumption."  True,  but 
keep  in  mind  that  Robert  F.  Mager  (1962)  has  tredned  thousands  of  people 
to  write  behavloreLly  stated  training  objectives  but  never  required  they 
"write"  an  objective,  as  part  of  his  training. 

The  enormous  amomt  of  research  on  evaluating  teacher  effectiveness,  most 
of  which  has  been  conducted  in  public  school  and  university  settings ,  is , 
for  all  practical  purposes,  entirely  consistent  in  results:  we  simply  can't 
do  it.  It  is  difficult  to  reason  that  we  are  any  different  in  our  serivce 
school  instructor  training  courses. 

One  possible  approach  now  being  tried  within  the  Instructor  Training  Course 
in  the  Adjutant  General  School  is  lllxastrated  at  Appendix  ft.  In  addition  to 
each  student  prepeuring  and  presenting  a  "^0  minute  lesson",  they  also  pre- 
pare  and  give  their  students  a  test.  The  student  trainee  then  evaluates 
the  results  in  terms  of  the  performtncea  demonstrated  by  his  "students".  As 
we  seem  to  have  leeumed  with  systems  engineering,  optimum  training  best  comes 
about  through  an  iterative  process:  a  tried-test-revision  kind  of  approach. 

We  ask  our  instructor  tredning  students  to  identity  what  changes  they  would 
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make  If  they  were  to  give  that  lesson  again.  We  have  shifted  our  evalu^ 
tion  emphasis  in  the  instructor  training  course  away  from  the  process  and 
toward  the  product.  We  have  no  illusions  that  this  is  an  objective  evalua¬ 
tion  means.  It  isn't.  It  seems  to  have  more  objectivity  however  than 
did  our  effoz*t8  to  assess  the  processes  of  instruction  (the  means,  methods, 
media  and  instructor  behaviors).  As  we  do  intend  that  our  stiidents  be  able 
to  do  something  as  a  result  of  the  training,  why  not  use  that  as  a  basis 
for  evaluating  Instructor  effectiveness. 

The  formal  body  of  psychological  research  literature  (social,  industrial, 
educational,  managerial,  etc.)  offers  a  vastly  rich  basis  for  clues  to  iden¬ 
tifying  supervisory,  leadership,  and  communicative  skills  which  can  be  ac¬ 
quired  through  training  and  which  do  transfer  into  relevant  criterion  situ¬ 
ations.  This  body  of  literature,  althou^  in  the  main  concerned  with  the 
basic  goals  of  science— description,  prediction,  and  control;  and  possibly 
Just  at  the  beginnings  of  the  descriptive  state— could  help  identify  those 
skills  which  are  trainable. 

Along  with  this  search  for  trainable  skills  another  set  of  questions  will 
rise.  Should  we  teach  a  particular  skill  Just  because  it  has  been  donon- 
strated  to  provide  pay-off  in  a  particular  research  situation?  For  exas^le, 
the  researdi  literature  suggests  that  an  individxial's  basic  personality  dis¬ 
position  of  authoritarianism  cannot  be  changed  but  that  his  leadership  style 
can  be  changed  to  non-authoritarian  means.  As  Likert  (1961)  phrases  this 
notion:  the  general  principle  followed  by  hi^-produclng  managers  is  the 
"principle  of  stg>portlve  relationships  (p.  103).”  Others  have  labeled  this 
svqpervisory  approach  a  "person  centered"  leadership  style. 

Do  we  train  our  personnel  in  such  non-authoritarian  leadership  skills;  should 
we,  what  are  the  consequences,  do  some  leadership  situations  demand  authori¬ 
tarian  approaches,  can  we  train  the  skill  of  identifying  which  situations  re¬ 
quire  a  particuleo*  style?  Do  we  actually  know  enoi:gh  to  make  such  decisions? 

We  have  not  had  to  face  such  questions  because  we  haven't  atteitq)ted  to  identify 
the  outcomes  of  our  training:  our  measurements  sa*e  too  siibjective  and  relative. 
We  will  raise  such  questions  by  trying  to  operationalize,  make  objective,  or 
identify  the  observables  within  leadership  and  communicative  skills.  The  pro¬ 
cess  of  searching  for  the  observables,  through  successive  ^proxlmations , 
could  bring  closer  to  worthwhile  and  meaningful  outcomes.  We  will  have 
lost  nothing  by  the  effort  for  neither  the  means  nor  the  ends  of  our  present 
training  in  leadership  and  commtinicative  skills  is  understood. 
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EVALUATION  OF  TEST  RESULTS  FOB 
SO  MINUTE  PRESENTATION 

1.  Analysis  of  test  results  sbovs : 

YES  HO 

A.  Evidence  of  possible  deficiencies 

in  training  _  _ 

B.  Evidence  of  possible  overtraining  _  _____ 

2.  Which  of  the  following  actions  could  be  taken  to  iiQ>rove  student 
performance?  (Check  eO.1  appropriate  actions.) 

A.  Evaluation 

_  1,  Change  the  test  instrument. 

____  2.  Change  the  directions  on  the  test. 

_____  3**  Change-  the  verbal  instructions  for  test  administration. 

B.  Instruction 

_ 1.  Select  different  main  teacdiing  points. 

_ 2.  Select  dlffemet  st^porting  materleLLs. 

_  3.  Select  different  Instructional  methods. 

_  U.  Select  different  instructional  media. 

____  Change  the  sequence  of  InstructloneJ.  material. 

3.  If  actions  taken  to  Inqprove  performance  could  not  all  be  accomplished 
prior  to  the  next  time  the  instruction  and  tests  are  given,  which 
changes  must  be  made  k-ecause  they  are  the  most  important?  Briefly 
explain  your  answer. 


APPENDIX  A  to:  Evaluation  of  Leadership  and  Communicative  Skills 


MORK^P  CONSULTANT’S  OBSERVATIONS 


DR.  T.  0.  JACOBS 

The  function  of  the  Quality  Control  Mechanism  in  Training  Technology  is 
to  provide  answers  J^o  the  following  two  questions: 

1.  What  are  the  real  world  training  requirements  an  institution  must 
satisfy? 

2.  How  well  is  this  Institution  satisfying  those  requireioents? 

Implicit  in  the  second  question  is  the  assumption  that  some  regular  change 
may  be  necessary.  Therefore,  a  correlary  to  the  second  q\iestion  is,  "What 
change  is  needed,  and  what  direction  should  be  taken?" 

The  four  major  events  of  the  half-day  program  have  been  oriented  toward  one 
or  both  of  these  primary  questions. 

THE  MILITARY  OCCUPATIONAL  DATA  BANK. 

User  conments  about  MOOB  have  been  aid>ivalent  in  the  past.  One  problem  is 
that  MODS  output  may  not  be  at  a  level  useful  for  training  develoiment, 
especially  in  soft  skill  areas  where  a  succinct  list  of  task  statements  may 
not  be  available.  However,  the  changes  anticipated  in  the  type  of  data  in 
the  MODS,  and  plans  for  its  utilization,  seem  excellent.  The  implications 
of  having  MODB  entries  conqpatible  with  CODAP  programs  are  enormous.  In 
research  on  active  duty  assignments  of  knay  ROTC  graduates,  Scott,  Powers, 
and  Sucansky,^  found  520  different  principle  duties  to  have  been  assigned 
to  a  total  of  1,898  ROTC  Graduates  serving  in  10  different  branches. 

These  initial  duty  assignments  covered  an  unbelievable  variety  of  jobs  and 
no  one  duty  was  reported  by  more  than  12%  of  the  total  sas^le.  We  are 
currently  studying  duty  positions  held  by  eight  key  MOS  in  four  combat  arms 
We  anticipate  that  a  similar  outcome  will  be  found. 

This  poses  an  almost  insurmountable  problem  to  the  trainer.  Changes  in  the 
MODB  that  we  have  heard  described  today  may  well  offer  solutions  to  this 
problem.  For  exaflq>le,  use  of  CODAP  programs  may  make  streamlining  of  the 
MOS  structure  possible,  which  would  siiq>lify  both  the  problems  of  school 
training  and  quality  control.  Changes  may  also  facilitate  movement  in  the 
direction  of  increasing  responsibility  for  MOS  training  within  units. 


1 

Scott,  Joseph  W.,  Powers,  Theodore  !{.,  and  Sucansky,  Paul. 

An  analysis  of  initial  duty  assignments  qf  Army  ROTC  Graduates,  Technical 
Report,  66-16,  Human  Resources  Research  Organization,  Alexandria,  Virginia 
October  1966. 
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THE  POST  GRADUATE  QUESTIONNAIRE. 

This  is  8  good  i^roach  to  Quality  Control,  which  should  be  continued. 
Especially  for  hard  skills,  such  questionnaires  provide  effective  diagnosis 
of  both  the  quality  of  the  training  and  the  extent  to  which  new  field  re> 
quireswnts  nay  be  inpacting  on  the  overall  effectiveness  of  the  training 
itself.  However,  it  appears  that  Post  Graduate  Questionnaires  may  be 
relatively  more  limited  in  Soft  Skill  areas.  Specifically,  it  is  hard  to 
see  how  they  can  be  of  great  diagnostic  value  in  soft  skill  areas  that  have 
not  been  Systems  Engineered.  That  is,  the  school  may  not  know  how  to  ask 
questions  about  the  quality  of  its  training,  and  Conmanders  in  the  field 
may  not  know  how  to  provide  that  which  is  meaningful  to  the  school . 

USER  REACTION  PANEL. 

This  technique  appears  to  be  excellent  for  internal  quality  control,  when 
zhe  school  is  concerned  about  the  relative  effectiveness  of  its  use  of 
training  technology.  However,  caution  must  be  observed  in  the  use  of  such 
panels.  First,  students  are  not  trained  trainers,  therefore,  their  cosments 
are  not  to  be  taken  at  face  value.  If  a  coiq;>etent  training  specialist  is 
available  to  listen  to  and  analyze  the  panel's  output  this  would  seen  a 
quite  effective  way  of  monitoring  pedagogical  matters.  As  a  second  caution, 
the  possibility  of  bias  in  any  given  panel  suggests  that  several  should  be 
drawn  from  a  class  whenever  such  an  assessment  is  being  performed.  Finally, 
the  school  or  the  moderator  is  probably  a  major  variable.  The  panel  we  saw 
today  was  very  effectively  conducted. 

CURRENT  APPROACHES  TO  LEADERSHIP  ASSESSMENTS. 

The  topic  of  leadership  has  very  high  salients  in  the  military  establishment, 
and  rightly  so.  Leadership  is  what  makes  for  superior  organizational  per¬ 
formance.  In  almost  any  situation,  the  more  leadership  there  is,  the  better 
the  organization  does.  But  there  are  questions. 

What  is  it? 

How  do  you  measure  it? 

How  do  you  develop  it? 

How  can  you  get  feeuback,  if  the  developing  institution  nor  the 
user  knows  how  to  verbalize  what  it  is? 

We  have  discussed  a  nuad)er  of  ways  of  assessing  leadership,  none  of  which 
are  by  any  means  perfect.  It  probably  would  be  a  mistake  to  reject  them,  for 
only  with  experience  is  it  possible  zo  learn  how  effectively  an  assesment 
system  is  operating. 

There  are  loajor  cautions  that  should  be  observed.  It  would  seem  that  any 
school  which  presently  is  assessing  leadership  and  is  using  that  assessment 
as  a  permanent  entry  on  the  individual's  records  (or  worse  still,  selecting 
for  graduates  on  the  basis  of  that  assessment)  may  be  causing  gross  injustices. 
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Thus  far,  only  peer  ratings  appear  to  show  satisfactory  (but  not  very) 
validities  on  a  long-range  basis.  This  probably  is  because  peer  ratings 
measure  intei^rsonal  competence,  at  least  to  some  extent.  Interpersonal 
coapettoice  is,  in  all  likelihood,  one  of  the  key  leadership  skills.  How¬ 
ever,  when  one  goes  beyond  peer  ratings,  validities  are  much  snre  tenuous. 
Situational  tests  have  a  strong  tendency  to  become  unreliable  over  a  period 
of  years  (the  problem  is  continued  standardization  of  observers) .  Piper 
and  pencil  tests,  especially  personality  tests,  may  be  transparent  and 
fakable.  When  decisions  are  made  about  the  disposition  of  a  student  on  the 
basis  of  scores  on  a  paper  and  pencil  instrument,  and  where  the  instnmwnt 
is  fakable,  the  highest  scoring  students  may  simply  be  the  best  fakers. 

Thus  far,  I  know  of  no  substantiated  relationship  betwem  faking  and  leader¬ 
ship. 

Research  is  continuing  on  leadership  assessment,  however.  A  leadership 
assessment  center  is  being  planned  at  Fort  Banning,  and  an  ejqperiisent  will 
be  conducted  to  learn  how  effectively  leadership  can  be  assessed,  and  de*- 
veloped/predicted  through  the  use  of  this  approach.  While  it  obviously  is 
going  to  be  necessary  to  settle  first  on  what  leadership  is,  the  assessment 
center  approach  probably  is  the  most  promising  right  now,  with  a  potential 
for  wide-spread  implementation  if  it  proves  effective. 

SUMMARY. 

In  summary,  it  appears  that  progress  is  being  made  toward  the  development 
of  effective  te(^iques  for  soft  skill  Quality  Control.  However,  such  means 
do  not  presently  exist  except  in  primitive  form.  While  the  use  of  these 
primitive  forms  should  not  be  discouraged,  neither  should  consequences  to 
students  be  allowed  until  much  more  sophisticated  approaches  have  been 
developed. 

It  would  seem  that  an  essential  prerequisite  for  a  quality  control  system, 
is  a  capacity  for  systems  engineering  these  soft  skills.  Once  reliable 
techniques  are  developed  for  this  purpose,  it  should  be  a  relatively  sinpler 
next  step  to  develop  the  measurement  techniques  essential  for  effective 
quality  control  systems. 
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SCHEDULE 

FOR 

SOFT -SKILL  TRAINING  DEVELOPMENTS  WORKSHOP 

13  December  1972 
Hinman  Hall  -  Room  169 

Chairman;  Dr.  Neal  B.  Andregg 

Educational  Advisor,  USAMPS 

Resource  Consultant:  Dr.  Richard  W.  Burns 
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0800-0820 

1300-1320 

Process  Procedure  and 
Development  of  Soft  Skill 
Training 

Performance  Training  for 
Military  Police  Noncommis¬ 
sioned  Officers 

CPT  D.  W.  Garner 
USAMPS 

Mr.  Clarence  C. 
Jeter,  Jr,  USAMPS 

0820-0830 

1320-1330 

Discussion  of  Above 

0830-0850 

1330-1350 

Performance  Based  Teacher 
Education  -  A  Program  and 
Problems 

Dr.  R.  W.  Burns 
UTEP 

0850-0900 

1350-1400 

Discussion  of  Above 

0900-0920 

1400-1420 

Procedures  for  Implementing 
Soft-Skill  Training  in  CONARC 
Schools 

Dr.  J.  P.  Fry 
HUMRRO 

0920-0930 

1420-1430 

Discussion  of  Above 

0930-0950 

1430-1450 

An  Improved  Army 
Classification  Battery 

Dr.  M.  H.  Maier 
USA  Research 
Institute 

0950-1000 

1450-1500 

Discussion  of  Above 
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Schedule  for  Soft-Skill  Training  Developments  Workshop  -  Continued 


1000-1015  1500-1515  Break 

1015-1030  1515-1530  Field  Visitation  Program  Dr.  C.  O.  Gray 

Briefing  USAES 

1030-1045  1530-1545  Use  of  Television  and  Video  Dr.  N.  B.  Andregg 

Tapes  in  the  Training  of  USAMPS 

Polygraph  Examiners  at 
USAMPS 

1045-1050  1545-1550  Discussion  of  Above 

1050-1110  1550-1610  CONARC  Project  ABACUS 

A  Prototype  Computerised 
Training  System  for  the 
Army 

1110-1115  1610-1615  Discussion  of  Above 

1115-1130  1615-1630  Observations  and  Summary  Dr.  R.  W.  Burns 

UTEP 

1130  1630  Return  to  Auditorium 

for  Plenary  Session 


COL  G.  B.  Howard 
Product  Mgr 
USASCS 
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DR.  NEALB.  ANDREGG 


Dr.  Andregg  i«  currently  assigned  as  Special  Assistant  -  Educatioxial 
Advisor  to  the  Commandant,  U.  S.  Army  Military  Police  School,  Fort 
Gordon,  Georgia. 

He  received  a  BS  in  Education  and  a  MA  in  Mathematics  from  Ohio  State 
University  and  the  EdD  in  Educational  Psychology  and  Statistics  from 
Michigan  State  University.  Prior  to  his  present  assignment,  he  was  Director 
of  Evaluation  and  Professor  of  Education  at  the  Air  University,  Maxwell 
Air  Force  Base,  Alabama. 

He  is  a  member  of  Phi  Delta  Kappa,  the  National  Society  for  the  Study 
of  Education,  the  National  Council  on  Measurement  in  Education  and  several 
other  professional  organizations.  He  teaches  graduate  courses  in  Education 
for  the  University  of  Georgia  and  undergraduate  courses  in  mathematics  and 
education  at  Augusta  College. 
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SOFT-SKILL  TRAINING  DEVELOPMENTS 


SUMMARY 

DR.  NEAL  B.  ANDREGG 

The  seven  presentations  made  it  apparent  to  the  participants  that  much 
progress  has  been  made  during  the  past  year  in  soft-skill  training  develop¬ 
ments.  Performance  training  and  testing  in  NCOES  courses  at  USAMPS 
focus  on  problems  with  built-in  stress  situations,  individual  student  per¬ 
formance.  and  the  requirement  to  communicate  and  interact  with  others 
using  information  available  prior  to  the  application  phase. 

The  University  of  Texas  at  El  Paso  has  adopted  a  Performance-Based 
Teacher  Education  Program  which  has  many  new  and  interesting  features. 
Those  responsible  for  faculty  development  in  CONARC  schools  might  well 
consider  using  some  of  the  features  of  this  individualized  program. 

HumRRO  emphasized  a  problem-solving  approach  to  soft- skill  training  with 
seven  characteristics  which  they  have  been  able  to  try  out  at  the  Fort  Bliss 
Modem  Volunteer  Army  office  in  a  Performance -Counseling  Workshop.  This 
experience  may  have  a  direct  bearing  on  the  basic  characteristics  of  the 
Chemical  School  Advanced  Course  also  described  in  this  research  paper. 

The  Improved  Army  Classification  Battery,  a  ten-year  project,  should  con¬ 
tribute  greatly  in  placing  personnel  in  the  training  program  best  suited  to 
their  capabilities.  Personnel  who  enter  the  service  have  limited  work 
experience  and  little  technical  training.  A  new  selection  and  classification 
test  will  improve  the  selection  procedures  based  on  course  data  collected 
in  CONARC  schools. 

The  USAES  Field  Visitation  Program  should  prove  helpful  to  schools  inter¬ 
ested  in  validating  their  course  design  and  in  improving  other  school  functions 
such  as  doctrine  development,  unit  training,  support  for  ROTC  programs, 
and  development  of  training  literature  and  training  aids.' 

The  use  of  television  and  videotapes  in  the  training  of  polygraph  examiners 
at  USAMPS  should  provide  other  schools  with  ideas  for  a  new  application  of 
TV  which  aids  in  motivating  students  in  soft  skill  training.  Video  cassettes 
can  be  used  for  problem  presentations  reqtiiring  individual  and  group 
solution.  They  can  additionally  be  effectively  employed  for  testing. 

CONARC  Project  ABACUS,  to  be  concluded  in  August  1976.  is  a  prototype 
computerised  training  system  for  the  Army  which  combines  CAI  and  CMI 
functions.  Its  goal  is  a  suitable,  low  cost,  viable,  and  effective  hardware 
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•yat«m»  and  a  newly  developed  laiigyAgd'.yvat  will  facilitate  course  develop¬ 
ment  a^  provide  maximum  flexibility.  From  the  prototype,  design  speci¬ 
fications  can  be  prepared  t(;»^ee^i^e,i«ftdf39l  COVARC  schools  and  training 
centers.  Conferees  were  afforded  the  opportunity  to  observe  an  example 
of  this  system  in  action.  ,,,  .  ,  ^  . 
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CAPTAIN  DAVID  W.  GARNER 
United  States  Army  Military  Police  School 

CPT  David  Garner  is  a  career  Army  officer  with  over  8  years  of 
conunissioned  service.  He  has  served  in  a  variety  of  positions  in  both 
the  Infantry  and  Military  Police.  His  prior  assignments  include  duty  with 
mechanised  infantry  units  in  the  Zd  Armored  Division  as  Platoon  Leader, 
Support  Platoon  Leader,  Company  Commander  and  Battalion  S-3.  He  has 
held  assignments  as  Assistant  S-3  of  the  504th  MP  Battalion,  Commanding 
Officer,  Z18th  MP  Company  and  as  a  subarea  Provost  Marshal  while 
serving  in  the  Republic  of  Vietnam.  CPT  Garner  is  a  graduate  of  the 
Infantry  Officer  Basic  Course,  Airborne  School,  the  4th  Army  Motor 
Maintenance  Course  and  the  Military  Police  Officer  Advanced  Course.  He 
is  a  graduate  of  the  University  of  Texas  at  Austin  (BBA,  Marketing  Research, 
1964)  and  Texas  Tech  University  (MBA,  Management,  1970).  He  is  currently 
assigned  as  Chief,  Systems  Engineering  Branch,  Office  of  the  Deputy  Com¬ 
mandant  for  Education  and  Training  and  has  worked  extensively  with  the 
development  of  the  Military  Police  NCOES,  beginning  in  September  1970. 


PROCESS,  PROCEDURE  ifUD  DEVELOPMENT  OF  SOFi  SKILL  miNING 

or 

What  To  Do  When  You  Run  Out  Of  Excuses 


CPT  DAVID  W  GARNER 

United  States  Amy  Military  Police  School 


In  Deceiid)er  of  1969  (XWARC  directed  the  development  of  performance-oriented 
training  programs  for  noncommissioned  officers  within  each  Army  Career  Man- 
agMient  Field.  The  Military  Police  Sdiool,  not  unlike  its  sister  service 
schools,  was  particularly  iiqtressed  with  that  part  of  the  directive  which 
decreed  that  the  courses  would  be  systems  engineered.  In  retrospect  it's 
been  a  soul  searching  and  somewhat  trying  two  years  for  us.  Approxistately 
two  and  a  half  years  ago  USAMPS  had  managed  to  move  froai  reading  the  basic 
NCOES  letter  to  securing  copies  of  (X^ARC  Regulation  350-100-1. 

It  seems  that  now  we  are  all  immersed  in  the  "wonders"  of  systems  engineer¬ 
ing,  to  one  degree  or  another.  Virtui  .i>  every  person  present  has  probably 
suffered  from  the  "It  Can't  Be  Dc^^  jymdrome.  Over  the  course  of  time  the 
arguments,  especially  in  the  soft  skill  training  areas,  and  more  specifically 
the  "pec^le"  skill  areas,  have  centered  arotaul  a  lack  of  well-defined  doctrine 
and  terminology  related  to  job  requi^’ements .  When  tasked  to  train  leadership 
and  managerial  skills,  we  invariably  fall  back  to  "laundry  lists"  which 
categorically  outline  the  12  principles  of  good  lesson  plan  development  and 
7  essential  fundamentals  of  mss  kit  repair.  Comments  initially  received 
seemed  to  focus  on  an  apparent  unwillingness  to  cut  the  umbilical  cord  tying 
the  past  with  the  future.  Lesson  plans  and  tests  which  were  written  five 
years  ago,  revised  annually  and  provide  a  soft  security  blanket  are  no  longer 
valid.  What  happened?  Somebody  had  a  better  idea,  but  it  took  a  long  time 
for  anybody  to  listen. 

USAMPS,  not  unlike  others,  continued  to  talk  the  problem  to  death.  Everybody 
knew  yAiat  the  problem  was,  but  what  was  the  next  step.  Obviously  CONARC 
didn't  Man  us.  Our  curriculum  is  soft  skill  and  too  "people"  oriented.  Well, 
what's  the  next  course  of  action?  Our  guidance  from  above  is  all  hard  skill 
oriented  or  it's  tied  to  soft  skills  which  have  well-documented  procedural 
guidance.  Somehow  we  convinced  ourselves  that  we  had  completely  justified 
our  position,  but  CONARC  wasn't  buying  any  of  that.  We  cane  to  the  realizat¬ 
ion  that  we  had  just  walked  off  the  end  of  the  pier  and  fallen  into  sane  deep 
water. 

In  retrospect  the  "sink  or  swim"  concept  was  the  best  thing  that  could  have 

hai^ned.  Not  all  of  us  grew  water  wings - in  fact  there  were  a  lot  of 

us  who  got  washed  \qp  on  the  beach  and  hawled  off  to  the  dump.  Those  who 
didn't  "cross  over"  probably  didn't  for  a  couple  of  reasons. 
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(1)  A  siaple  lack  of  Botivation 

(2)  An  inherent  aistxust  of  diange 

\ 

(3)  The  firm  conviction  that  nobo<fy  knew  as  nuch  about  their 
instructional  area  and  methods  and  media  as  they  did 

(4)  An  inability  to  swim  in  water,  coupled  with  a  profound  belief 
that  swimming  is  best  learned  on  sandy  beaches,  not  in  water 

In  the  NCO  Basic  and  Advanced  Courses  we  followed,  in  the  limited  time  avail¬ 
able  the  process  of  job  identification,  developing  a  task  inventoiy  and  select¬ 
ing  tasks  for  training.  We  were  not  completely  satisfied  with  the  systems 
engineering  materials  that  were  developed  during  Step  3,  Training  Analysis, 
but  generally  they  made  more  sense  to  us  than  anything  we  had  tried  before. 
Initially  we  kept  hearing  the  words,  "leadership,  si^ervision,  management,  etc. 
The  inevitable  problem  was  how  to  get  a  haiulle  on  what  leadership  really  is, 
especially  in  terms  of  noncoaaissioned  officers.  We  started  talking  about 
this  with  noncommissioned  officers  in  our  branch.  Basically  they  explained 

idiat  it  meant  to  them - then  we  realized  it  encompassed  a  multitude  of 

profound  theories,  ranging  from  Maslow  to  McGregor  and  far  beyond.  Essent¬ 
ially  our  management  ^pes  told  us  that  there  were  all  sorts  of  component 
eleawnts  idiidi  make  t;g)  this  somewhat  loosely  defined  area  of  leadership. 
Basically,  in  a  leadership  situation  we  use  varying  amounts  of  each  of  these 
coaqwments.  Although  our  NCOs*  comments  were  a  little  more  direct  and  a  lot 
more  understandable  than  our  management  types'  it's  interesting  to  conpare 
the  two. 


lement  Calls  It 


NCOS  Call  It 


(Ij  Subordinate  Indebtedness 

(2)  Legal  Authority 

(3)  Information 

(4)  Communicative  Skills 

(5)  Status 

(6)  Motivation 

(7)  Self-Confidence 

(8)  Task  Eiqpertise 


(1)  Giving  the  Guy  a  Break 

(2)  Bucking-Up 

(3)  In-the-Know 

(4)  Getting -Through,  lUpping 

(5)  Prestige 

(6)  Hard-Charger 

(7)  Keeping  His  Cool 

(8)  Knowing  the  Territory 


Naturally,  each  of  the  elements  shown  interact  to  a  large  degree  with  one 
another  and  Influence  how  effectively  an  individual  performs  in  a  leadership 
situation. 


Notice  how  the  word  "situation"  sort  of  crept  into  the  discussion.  We  were 

learning - —now  we  were  getting  a  general  and  somewhat  foggy  impression 

of  what  was  floating  above  us. 
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We  were  ready  to  accept  this  business  of  leadership  components,  but  how 
did  we  train  men  in  this?  First  of  all,  we  could  lecture,  and  that's 
meant  with  all  the  bad  connotations.  Next,  we  could  review  lesson  plans 
and  discover  to  nobody's  surprise  that  there  will  be  the  inevitable 
laundry  list  idiich  reads  something  like  the  Boy  Scout  Oath.  All  right, 
if  we  talk  about  using  the  "tried  and  true"  vu>graph  techniques,  and  the 
associated  list  of  "principles  and  fundamentals"  how  do  we  get  the  student 
to  prove  to  us  he's  any  better  than  he  was  the  day  he  reported?  Well,  we 
could  submerge  him,  along  with  ourselves,  in  multiple  guess,  true/false, 
laundry  list  "evaluation"  instruments  and  other  foxms  of  perverted  s\ibject- 
ivity.  Of  course  this  seemed  to  be  about  the  same  approach  we'd  always 
used. 

Maybe  we  should  see  what  we  can  find  out  about  small  groiq>  relations  and 
what  goes  on,  both  foznally  and  informally.  Basically,  our  analysis  centered 
.on  studies  which  involved  groups  having  fairly  rapid  turn-over,  a  relatively 
small  number  of  people,  specific  responsibilities  to  be  carried  out  and 
defined  material  resources  available.  This  sounds  a  lot  like  the  normal 
situation  in  the  military  service,  and  it  is.  Well,  what  now?  Invariably 
in  these  situations  the  group  involved  looked  to  the  individual  having  the 
greatest  degree  of  technical  ability  for  their  guidance.  In  occasional 
instances  of  personality  deviations  the  tendency  of  the  group  was  to  reject 
technical  expertise  and  resort  to  some  modified  form  of  democratic  selection, 
usually  acclamation  based  on  raw  physical  strength,  demonstrated  taste  in 
clothes,  or  general  sex  qipeal. 

Since  we  now  know  that  e:q)ertise  in  the  job  is  the  first  basis  for  assuming 
a  leadership  position,  what  could  we  do  to  insure  that  our  NCO  held  his 
position  without  resorting  to  extra- legal  means,  to  include  corporeal  punish¬ 
ment,  after  others  in  his  group  started  to  acquire  some  ability  of  their  own? 
Well,  at  this  point  it  was  iqqparent  that  our  leader  better  be  able  to  cooamini- 
cate  with  others,  and  he  better  have  or  be  able  to  get  information  which  bears 
on  the  problems  he  will  be  faced  with  in  his  job.  Just  to  insure  he  is  ready 
to  come  to  grasp  with  John  Q.  Hardcase,  who  won't  accept  any  guidance,  we'll 
throw  in  exactly  what  our  NCO  is  and  is  not  authorized  to  do.  Interestingly 
enough,  we  wanted  to  build  self-CMifidence  and  by  design,  at  the  end  of  this 
whole  process  we  found  that  our  students  had  acquired  this  as  a  result  of 
reducing  leadership  to  manageable  instructional  tasks. 

In  essence  we  decided  that  these  courses  we  were  designing  had  to  focus  on 
the  tasks  the  student  will  perform,  but  moreover  we  had  to  work  into  these 
tasks  a  requirement  to  insure  the  student  could  conmnmicate,  use  information, 
and  know  exactly  how  much  authority  he  is  carrying  around  with  him.  We're 
emphasizing  leadership  as  a  direct  one-to-one  situation  in  which  an  influence 
attempt  is  directed  at  another.  If  the  receiver,  the  subordinate,  reacts  in 
a  manner  which  is  deemed  acceptable,  then  we  say  that  a  positive  leadership 
interaction  has  occurred.  This  is  not  to  say,  however,  that  one  reaction 
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caimot  be  sore  favorable  than  another,  nor  is  it  saying  all  leadership 
is  effective.  It  is  entirely  possible  to  have  a  favorable  reaction,  the 
subordinate  carries  out  the  assignaent,  with  less  effective  leadership 
Methods  applied.  This  most  frequently  occurs  when  the  legal  authority 
aspect  of  leadership  is  the  coaqwnent  relied  iqpon.  The  subordinate  reacts 
to  a  less  desirable  authoritative  af^roach  when  a  participative  approach 
would  have  achieved  the  saae  end  result  without  creating  subordinate  ill- 
will. 

Froai  tiiese  concepts  we  devlqped  our  NCOES  courses  to  focus  on  three  critical 
leadership  areas: 

(1)  Technical  Expertise 

(2)  CoBwunicative  Skills 

(3)  Derivation  and  Use  of  Information 

Our  performance  training  and  testing  focuses  on  situations  with  built-in 
stress,  live  individual  student  performance,  and  the  requirement  to  communi¬ 
cate  and  interact  with  others  using  information  available  prior  to  the  con¬ 
duct  of  the  application  phase. 


MR.  CLARENCE  C.  JETER,  JR. 
United  States  Army  Military  Police  School 


Mr.  Clarence  C.  Jeter,  Jr. ,  is  a  member  of  the  U.  S.  Army  Military 
Police  School  staff.  His  experience  includes  that  of  Training  Instructor 
in  the  Cryptographic  Operations,  Radio  Relay  and  Carrier  Eqmpment 
Repair  and  Instructor  Methods  Courses;  Training  Supervisor,  Assistant 
Chief,  Operations  Division  and  Section  Chief  of  the  Testing  and  Quality 
Control  Section  of  the  Southeastern  Signal  School.  He  is  currently  assigned 
to  the  Office  of  the  Deputy  Commandant  for  Education  and  Training  as  an 
Educational  Specialist.  Mr.  Jeter  attended  Furman  University,  College  of 
St.  Joseph  on  the  Rio  Grande  and  the  University  of  Georgia,  Augusta 
Extension;  he  graduated  from  Augusta  College  with  a  BBA  and  Georgia 
Southern  College  with  a  MEd.  Mr.  Jeter  lives  in  North  Augusta,  South 
Carolina,  with  his  wife  and  two  children. 
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PERFORMANCE  TRAINING  FOR  MILITARY  POLICE  NONCOMMISSIONED  OFFICERS 


NR.  CLARENCE  C.  JETER.  JR. 

United  States  Army  Military  Police  School 


Once  it  was  determined  that  the  United  States  Army  Military  Police  (USAMPS) 
would  follow  the  letter  and  the  spirit  of  the  systems  engineering  concept 
many  interesting  training  problems  developed  in  the  soft  skill  area.  The 
question  that  we  finally  became  engrossed  in  was  based  on  how  to  train  Law 
Enforconent  Noncommissioned  Officer  Basic  students .  How  could  the  school 
develop  learning  situations  which  would  transfer  to  the  field?  How  could 
a  large  number  of  realistic  situations  be  developed?  How  could  a  simulated 
job  environment  be  created  within  the  existing  training  facility? 

The  first  guideline  established  was  that  tasks  performed  in  the  same  environ¬ 
ment  would  be  grouped.  The  resulting  sequence  was  to  have  military  police 
post,  camp,  and  station  operation  tasks,  military  police  company  adainistra- 
tion  tasks,  and  MP  tactical  operation  tasks  grouped  as  the  broad  functional 
areas.  This  arrangement  later  allowed  functional  organizational  delineation 
within  the  Noncommissioned  Officer  Group.  The  training  objectives  were 
analyzed  and  found  to  be  directed  toward  three  instructional  goals  where  the 
student  must  learn  to:  solve  problems  and  make  decisions;  perform  program 
and  operational  planning;  and  display  attitudes  appropriate  to  his  position. 
The  size  of  the  class,  the  ty|>e  of  classroom  or  training  area,  and  the 
methods  of  instruction  were  dictated  by  this  analysis  and  the  philosophy  of 
performance  oriented  training. 

Classrooms  were  designed  to  support  practical  exercises  in  two  of  the  three 
functional  areas  by  environmentally  simulating  the  actual  job  conditions  at 
a  military  police  station  and  an  orderly  roan.  The  physical  layout  includes 
areas  for  four  student  and  eight  student  work  groups,  as  well  as  for  the 
full  class.  Optimum  class  size  was  determined  to  be  16  students  with  a 
maximum  class  size  of  20.  Telephones  were  installed  at  each  student  prac¬ 
tical  exercise  position;  a  switchboard  was  set  iq>  for  instructions,  carrels 
were  made,  and  tables,  chairs,  office  siqpplies  and  manuals  were  provided 
for  each  student.  Post  maps,  notification  diarts,  and  military  police  station 
standing  operating  procedures  were  prepared  and  printed  so  that  normal  job 
tools  would  be  available.  Speakers  and  noise  tapes  were  included  in  the 
training  area  to  simulate  as  much  of  the  desired  real  life  environment  as 
possible.  While  the  classroom  falls  short  of  the  real  orderly  room  or  actual 
military  police  station,  it  is  a  far  cry  from  the  regular  "dressed  vq)"  lecture 
room  that  had  preceded  it. 


With  the  classroom  arrangesients  cared  for,  the  next  step  was  to  identify  and 
develop  outside  training  areas  to  support  tactical  instruction  and  traffic 
accident  investigation  training.  Fixed  checkpoints  were  established  to  train 
for  population  and  resource  control  operations  and  suitable  areas  were  located 
tc  support  mobile  checkpoints.  Cars  were  bought,  accidents  staged,  and 
scripts  were  written  for  support  personnel  playing  parts  in  accident  investi¬ 
gation.  Other  training  objectives  requiring  an  outside  training  area  were 
planned. 

The  problem  solving  techniques  selected  to  train  in  soft  skill  areas  led  a 
small  number  of  instructors  to  the  giant  task  of  developing  large  nuBd)ers 
of  realistic  training  problems.  It  was  felt  that  unless  the  problems  were 
"real"  that  the  chance  of  transfer  would  be  small  and  notificational  edge 
would  be  lost.  The  instructors'  problem  surfaced  at  the  same  time  a  Military 
Police  Advanced  Officer  class  was  finishing  its  training  cycle.  These  officer 
students  had  about  fcnir  or  five  years  of  ejqperiences  which  related  to  the 
military  police  nonconmissioned  officer  job.  It  was  decided  that  this  group 
could  n^e  a  valuable  contribution  to  the  course  development  process  by 
writing  down  typical  situations  and  solutions  whcih  they  actually  encountered 
as  military  police  operation  officers,  military  police  duty  officers,  and 
assistant  provost  marshals .  Within  one  week  over  4S0  probloas  were  received 
by  the  instructors.  After  quality  control  action  was  completed  there  were 
about  150  solid,  real,  usuable  training  situations  on  hand  for  use  within 

the  military  police  post,  camp,  and  station  operations  segment  of  the  course. 
These  realistic  situations  are  used  for  testing  as  well  as  training.  The 

instructors  had  to  fall  back  the  regular  methods  of  problem-development  for 
the  other  segments  of  the  course;  long,  exhausting  work. 

The  Military  Police  School  feels  that  a  major  element  in  the  successful 
leader  is  die  ability  to  communicate.  Plans  were  made  to  insure  that  the 
students  had  to  communicate  with  the  types  of  people  they  would  be  con¬ 
fronted  with  on  the  job.  These  plans  did  not  include  subject  matter  blocks 
of  instruction  labeled  "Communication  Skills";  instead,  communication  skill«' 
are  taught  in  context  with  the  job  at  hand.  An  example  of  this  is  desk  ser¬ 
geant  job  which  includes  such  tasks  as  receiving  complaints,  determine 
authority  and  jurisdiction,  and  make  emergency  notifications.  The  desk  ser¬ 
geant  has  to  ccrnmninicate  with  almost  every  form  of  life  and  be  able  to  per¬ 
form  the  procedural  parts  of  the  tasks  at  the  same  time.  The  practical 
exercises  developed  for  this  area  of  the  course  include  the  student  talking 
over  the  phone,  taking  information  from  persons  at  his  desk,  and  giving 
instructions  to  subordinates;  all  within  the  framework  of  solving  job  related 
problems.  Just  to  insure  the  complete  environment,  the  noise  level  in  the 
classroom  is  maintained  through  tape  recordings  of  appropriate  background 
sounds.  The  Military  Police  School  thinks  that  if  the  student  experiences 
this  training  and  passes  the  performance  tests  related  to  this  instruction, 
he  leaves  this  program  at  a  reasonable  level  of  competence. 
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The  requirement  for  the  student  to  learn  to  display  certain  attitudes  during 
the  performance  of  various  tasks  has  caused  some  interesting  instructional 
situations  to  be  developed.  There  is  not  much  trouble  learning  to  write  a 
ticket  for  some  traffic  violation  when  all  it  amounts  to  is  taking  Infomation 
from  some  practical  exercise  work  sheet  and  placing  it  in  the  right  block 
while  sitting  in  a  classroom.  Try  the  same  task  when  the  information  giver 
is  crying,  angry,  or  drunk.  The  student  investigates  accidents  where  the 
drivers  of  the  cars  involved  begin  fighting  or  give  conflicting  views.  In 
community  relations  areas  the  learners  are  boad^arded  with  emotion  laden 
insults  so  that  they  can  learn  to  handle  their  feelings  and  their  job  at  the 
same  time.  Through  this  instruction  students  are  expected  to  act  as  profes- 
sional  law  enforcement  officers  and  competent  noncommissioned  officers.  The 
student's  first  response  to  these  situations  is  that  they  are  part  of  a  light 
game  but  as  training  goes  on  the  "game"  becomes  heavy  enough  to  cause  many  a 
stressful  moment.  Note  should  be  taken  that  there  is  no  block  of  instruction 
called  "attitude  Shaping":  this  training  requirement  is  spread  across  the 
objectives  by  being  addressed  wherever  it  can  be  considered  a  real  part  of 
the  job.  The  course  supervisors  and  instructors  have  been  very  cautious 
about  not  putting  stress  in  situations  that  actually  do  not  call  for  it. 

The  danger  is  selecting  this  approach  to  instruction  is  that  the  student 
material  will  be  flat  and  uninteresting  if  it  is  too  mundane  or  it  can  be 
a  joke  if  the  students  perceive  situations  as  fantasies.  The  noncommissioned 
officer  student  is  generally  well  grounded  in  his  job  and  is  quick  to  dis¬ 
criminate  between  the  real  and  the  unreal.  Eiqperience  with  advanced  and 
basic  officer  training,  as  well  as  the  noncommissioned  officers,  has  shown 
that  this  factor  gains  iiiq>ortance  as  the  job  background  of  the  student 
increases . 

The  noncoBunissioned  officer  education  program  in  the  Military  Police  School 
has  produced  gratifying  results.  The  testing  e'jfort  is  providing  informa¬ 
tion  which  is  usable  in  making  the  necessary  adjustments  in  the  systems 
engineered  program.  Current  performance  examinations  are  a  far  cry  from 
the  old  multiple  dioice  or  essay  tests  relied  on  in  the  past.  Field  feed¬ 
back  has  been  good  and  it  seems  the  graduate  is  being  well  received.  Many 
of  the  positive  aspects  of  this  track  have  been  picked  iq)  by  related  courses 
and  it  served  as  a  prototype  for  other  Noncommissioned  Officer  Educational 
System  programs  at  the  Military  Police  School.  Problems  continue  to  show 
(q>  during  normal  operations  and  they  are  dealt  with  as  a  matter  of  course. 

The  most  interesting  factor  coming  out  of  systmns  engineering  of  soft  skill 
courses  is  just  how  visible  real  training  problems  can  become.' 
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The  terms  performance -based  learning  or  competency -based  learning 
are  synonomous  designations  for  a  new  movement  in  American  Education. 
Change  has  been  described  as  one  of  the  sure  things  in  life,  but  public 
education  at  all  levels  has  not  been  known  as  an  institution  responsive  to 
change.  Although  at  first  glance  "schooling"  has  changed  from  the 
"little  red  schoolhouse"  of  the  1800's  to  "glass-brick  plants  wired  for 
sight  and  sound"  in  the  1900's,  the  basic  learning  process  has  remained 
frustratingly  rigid.  Even  in  the  military  and  industrial  segments  of  our 
society,  the  educational  or  training  endeavor,  although  more  responsive  to 
cultural  or  immediate  demands  than  public  education,  still  falls  short  of 
the  idealized  expectations  of  educators  or  trainers. 

Recently,  signs  of  a  rather  sweeping  reform,  most  commonly  called 
competency -based  education,  has  captured  the  attention  of  all  segments 
of  our  society.  This  movement  is  on  the  verge  of  practical  realization 
because  of  three  forces:  social  forces  --  the  demand  for  relevancy, 
effectiveness,  accountability;  technological  forces  --  management  and 
systems  strategies,  communication  devices,  print  and  non-print  innova¬ 
tions;  and  behavioral  science  forces  --  new  insights  into  learning  theory, 
performance  objectives  and  individualized  instruction. 

Exactly  what  is  competency-based  education?  It  probably  should  not 
be  defined  and  it  positively  cannot  be  described.  However,  it  seems  to 
have  in  all  of  its  variations  two  major  characteristics:  (I)  it  is  goal  or 
objective  oriented  and  (2)  it  emphasizes  the  individual  learner.  However, 
these  two  characteristics  of  learning  are  far  from  new;  in  fact,  they,  by, 
necessity,  have  characterized  human  learning  since  humans  first  "became.  ' 
What  then  makes  competency -based  education  a  new  movement?  It  is 
best  viewed  as  a  present-day,  systematic  attempt  to  implement  directed 
learning  in  an  effective  manner  --a  goal  long  sought  but  to  date  difficult 
to  achieve.  Now,  because  of  increased  understanding  of  how  individuals 
learn  and  technological  innovations,  competency-based  education  has  a 


realistic  chance  to  succeed  where  other  programatic  attempts  in  the 
same  direction  have  failed. 


Competency-based  education  does  these  things: 

1.  It  defines  the  probable  results  of  learning  in  specific  performance 
terms. 

2.  It  demands  that  the  learning  process  be  accountable  for  the 
described  performances,  and 

3.  It  recognizes  and  implements  learning  as  an  individiialized 
process. 

Associated  with  the  competency -based  movement  are  such  things  as: 
learning  packages,  teaching  machines,  consortial  decision  making,  and 
criterion-referenced  tests.  These  features  are  not  inherent  characteris¬ 
tics  but  rather  means  being  employed  to  reach  competency -based  goals. 

With  this  introduction,  I  wish  to  devote  the  rest  of  this  paper  to  a 
brief  description  of  a  specific  application  of  competency -based  learning; 
namely  The  University  of  Texas  at  El  Paso’s  Performance -Based  Teacher 
Education  Program.  The  project  started  in  the  summer  of  1970  when 
the  University  was  invited  to  participate  in  the  development  of  experimental 
performance -based  programs  in  Texas.  The  U.  T.  El  Paso  component 
proposed  an  experimental  design  which  was  to  be  a  model  applicable  to 
any  educational  level.  The  major  features  of  the  experimental  program 
were: 

1.  A  product  analysis  to  help  determine  the  appropriate  behavioral 
objectives. 

2.  Professional  staff  retraining  and  development  of  objectives. 

3.  Development  of  instructional  modules  in  three  core  areas: 
professional-technical,  psycho-personal  and  social  cultural. 

4.  Individualized  (self-paced)  instruction. 

5.  Specialized  field  experiences. 

6.  Group  seminars. 

7.  Computerized  management  system.  ^ 
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One  additional  purpose  of  this  project  from  the  University *8  viewpoint  was 
to  start  doing  what  we  believe  others  could  do.  We  wanted  to  show  teachers 
something  about  how  they  could,  in  turn,  be  teaching.  A  further  purpose 
from  the  State  of  Texas  point  of  view  was  to  establish  the  feasibility  of 
and  gain  experience  with  performance -based  teacher  training  with  some 
expectation  that  in  the  near  future  such  programs  would  characterize 
all  of  teacher  training  within  the  state. 

Let's  take  a  further  look  at  each  of  the  seven  major  features  of  the 
U.  T.  El  Paso  experimental  program  which  is  now  being  implemented  in 
the  classroom  in  its  third  year.  Objectives  and  job-task  analyses  are 
not  new  to  the  military  and  they  are  not  novel  ways  to  decide  on  appro¬ 
priate  objectives  for  a  training  program.  Perhaps  product  analyses  have 
also  been  functional  in  program  designs  but  they  are  less  well  known  and 
less  widely  used.  In  one  sense,  a  product  analysis  goes  a  step  further 
than  the  more  traditional  job-task  analysis.  In  our  program  the  product 
analysis  went  like  this: 

1.  Study  the  educational  product  --  the  learner  --to  determine  in 
so  far  as  possible  the  performances  expected  of  the  product  as 
a  result  of  learning. 

2.  Infer  from  step  1  the  teacher  performances  which  would  be 
required  to  produce  the  product. 

3.  Assess  the  performance  repertoire  of  each  entering  teacher 
training  intern. 

4.  The  required  performances  defined  in  step  2  minus  the  performances 
identified  in  step  3  become  the  individualized  performances  expected 
as  a  result  of  learning  in  the  performance -based  program. 

Being  aware  of  the  limitations  of  time,  available  techniques  and 
available  students  for  a  product  analysis,  a  supplementary  and  to  some 
degree  a  back-up  method  for  deriving  program  objectives  was  also  under¬ 
taken.  This  second  feature  of  our  program  also  was  intended  to  help  meet 
a  concurrent  objective  of  the  project;  namely,  staff  retraining  and  develop¬ 
ment.  In  this  instance,  each  staff  member  who  regularly  taught  a  course 
within  the  traditional  teacher -training  program  was  asked  to  write  an  idealized  list 
of  terminal -behavioral  objectives  for  that  particular  course.  They  wrote 
the  objectives  after  a  training  session  on  "Creating  and  Writing  Objectives". 

A  group  of  experts  reviewed,  rewrote,  selected  and  finally  sequenced  all 
the  objectives  (those  developed  "in  house"  and  those  resulting  from  the 
product  analysis)  into  one  comprehensive  set  for  the  program. 
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Once  the  scope  and  sequence  of  the  prograun  was  determined,  the 
pre-planning  of  the  instructional  process  itself  was  undertaken.  It  was 
decided  to  develop  instructional  modules  following  a  common  format 
where  possible.  This  third  feature  of  the  program,  the  modular  design, 
allowed  the  program,  in  its  process  phase,  to  be  self-paced  with  alterna¬ 
tive  methods  for  learning.  There  was  no  insistence  that  each  module  be 
of  any  given  length  or  design  but  rather  that  the  format  be  adapted  to  the 
objectives  selected  for  that  module.  Whereas  the  majority  of  the  modules 
were  similar,  especially  in  the  professional -technical  core,  diversity 
did  occur,  mainly  in  the  social-cultural  segment  and  to  some  degree  in 
the  psycho-personal  core.  The  suggested  format  for  the  instructional 
modules  consisted  of  these  features: 

1.  Title  page 

2.  Objectives 

3.  Overview 

4.  Pretest 

5 .  Rationale  >» 

6.  Instructional  alternatives 

7.  Post  test 

8.  Resources^ 

The  important  features  of  the  modular  design  which  helped  implement  the 
performance  concept  and  individualized  instruction  were  the  objectives, 
the  tests  and  the  instructional  alternatives. 

This  leads  to  the  fourth  feature  of  the  U.  T.  El  Paso  Performance - 
based  program,  individualized  instruction.  The  term  "individualized 
instruction"  has  many  meanings,  a  conclusion  I  have  reached  after  attempting 
to  read  about  it  and  observing  many  instances  of  its  purported  application 
in  the  field.  It  is  remarkable  how  many  educators  confuse  individualized 
instruction  with  independent  study.  The  latter,  independent  study,  could 
and  sometimes  does,  merely  mean  studying  by  yourself  --  alone  --  not  in 
a  group.  The  intent,  however,  of  individualized  instruction  is  to  effect 
a  "best  fit"  between  the  individual  learner  and  the  process  of  learning. 

^For  a  more  descriptive  treatment  of  an  instructional  module  design  ^ 
see  R.  W.  Burns,  "An  Instructional  Module  Design,  "  Educational  Technolo 

Vol.  12,  No.  9,  September  1972.  ^ 
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Essentially  the  concept  of  individualization  encompasses  three  vari¬ 
ables:  time,  method  (how)  and  content  (what  or  objectives).  Time 
as  a  variable  is  a  must  in  individualized  learning.  All  evidence 
shows  that  learners  do  indeed  have  unique  learning  speeds  and, 
further,  that  given  time  the  vast  majority  of  learners  can  achieve 
stated  objectives.  There  are  a  few  instances  where  exceptions  to 
time  as  a  variable  must  be  made  in  a  practical  sense;  especially 
where  cost-effectiveness  is  a  major  program  consideration.  Further, 
evidence  that  time  is  a  necessary  variable  is  found  in  the  true  mean¬ 
ing  of  the  commonly  expressed  phrase,  "a  slow  learner"  --  a  person 
who  can  learn  or  achieve  but  who  merely  needs  more  time  than 
generally  allowed  by  our  present  lock-step  educational  systems. 

The  second  essential  variable  in  individualized  instruction  in¬ 
volves  the  method  by  which  one  learns  or  the  "how".  It  is  safe  to 
say  that,  from  what  we  know  about  learning  as  an  interactive  process, 
no  two  learners  learn  exactly  in  the  saune  way  or  manner.  In  my 
previous  comments  about  the  instructional  module  design,  I  made 
reference  to  the  part  entitled  "instructional  alternatives"  as  being 
one  of  its  major  features.  The  intent  of  providing  instructional 
alternatives  in  a  modvile  is  to  implement  "individualization  of 
instruction"  by  making  available  to  the  learner  alternative  methods 
or  ways  in  which  he  may  attempt  to  attain  the  objective  of  the  module. 

In  practice,  the  alternatives  offered  represent,  whenever  possible, 
distinctive  instructional  situations  with  which  the  learner  can  interact. 
For  example,  one  alternative  could  appeal  to  the  visual  sense,  another 
to  the  aiidial,  another  to  the  tactile;  or  one  alternative  could  be  textual 
reading,  while  another  could  be  a  slide  tape  and  yet  a  third,  programmed 
instruction.  The  variations  in  alternatives  possible  are  only  limited 
by  the  module  designer's  imagination.  Here  is  a  list  of  thirteen  general 
types  of  instructional  alternatives,  each  of  which  has  many  variations: 

1 .  Film 

2.  Sound  Film 

3.  Slide  Tape 

s 

4.  Tape 


5.  Reading 

6.  Lecture 
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7.  Programmed  Initruction 

8.  On-the-job 

9.  Obaervation 

10.  Study-guide  question  plus  bibliography 

11.  CAl 

12.  Any  combination  of  the  above 

13.  Do  your  own  thing 

The  third  essential  variable  in  individualized  instruction  is  rarely 
permitted,  encouraged  or  provided  for,  and  that  is,  freedom  for  the 
learner  to  select  or  create  his  own  objectives.  Arguments  for  and 
against  student  selection  or  creation  of  objectives  can  be  made  and  in 
specific  skill  or  occupational  training,  as  frequently  found  in  military 
schools,  there  would  obviously  be  restricted  opportunities  for  learner 
choice  as  compared  to  areas  where  the  objectives  are  more  general. 

Returning  to  the  major  features  of  the  U.  T.  El  Paso  experimental 
program,  the  fifth  feature  dealt  with  speciedized  field  experiences. 
Traditional  teacher -training  programs  have  some  on-the-job  truning 
and  generally  include  one  or  more  courses  entitled  "practice  teaching". 
For  years  there  has  been  considerable  dissatisfaction  with  this  tradi¬ 
tional  approach;  mainly  because  it  was  felt  that  teacher-training  ^ould 
be  more  functional  if  there  was  integration  between  theory  and  practice. 
Field  experiences  as  conceived  in  new  programs  attempt  to  provide  a 
functional  relationship  throughout  the  whole  training  program  between 
University  classes  and  public  school  classrooms.  On-the-job  observa¬ 
tions  begin  early,  and  many  modules,  especially  in  the  psycho -personal 
core,  require  specific  field  experiences  as  part  of  the  module  process. 
Thus,  the  teacher  trainee  becomes  acquainted  with  the  readities  of 
teaching  early  enough  so  that  his  understanding  of  and  percoption  of  the 
need  for  professional  training  can  be  motivating  in  the  academic  phase 
of  his  training.  Additionally,  his  appreciation  of  the  school  system  and 
the  role  teachers  play  as  well  as  new  attitudes  towards  students,  pro- 
fessionatl  staff  and  prograun  are  formed  prior  to  teaching  rather  than 
following  employment. 
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A  sixth  major  feature  of  the  program  is  the  provision  for  scheduled 
group  seminars.  These  group  sessions  are  intended  to  function  as  pro- 
biem  solving  sessions  and  process  events  where  the  attainment  of 
affective  objectives  is  of  primary  concern.  Feedback  to  the  program  by 
the  trainees  is  also  a  probably  outcome  although  how  extensive  and 
effective  the  feedback  remains  to  be  seen. 

To  implement  the  program  by  way  of  record  keeping  and  feedback 
for  program  improvement,  a  computerieed  management  system  was 
designed.  Data  provided  by  the  system  will  allow  the  making  of  state¬ 
ments  about  and  drawing  conclusions  about: 

1.  Learner  progress  --  modules  successfully  completed. 

2.  Number  of  modules  being  utilized  at  any  point  in  time. 

3.  Elapsed  time  spent  interacting  with  each  module. 

4.  Laboratory  time  spent  interacting  with  each  module. 

5.  Number  of  modules  completed  by  individuals  per  unit  of  time. 

6.  Difficulty  of  modules  as  measured  by  elapsed  time  and  number 
of  recyclings. 

7.  Frequency  of  recycling  for  each  module. 

8.  Individual  trainee  pattern  of  modular  interaction. 

9.  Number  of  tests  administered  --  total  and  per  module. 

I  have  just  described  the  essential  features  of  the  U.  T.  El  Paso 
experimental  program.  In  practice,  minor  exceptions  and  deviations  from 
original  program  design  have  been  permitted  because  of  the  realities  of 
factors  as:  available  resources,  including  financial  support;  available 
staff:  space  requirements;  staff  personalities;  time  and  administrative 
constraints.  To  date,  all  the  modules  in  the  professional-technicad  and 
psycho-personal  cores  have  been  pretested,  some  of  them  revised  and 
they  are  now  available  for  use  both  within  the  experimental  program  and 
to  any  staff  member  who  may  desire  to  use  them  within  some  segment 
of  traditional  classroom  teaching.  In  this,  the  1972-73  school  year  the 
program  is  being  fully  implemented  for  the  first  time. 
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As  you  might  guess,  the  whole  project  has  not  escaped  the  plague 
of  expected  and  unexpected  problems  which  characterize  most  novel  educa¬ 
tional  efforts.  The  major  problems  to  date  have  been  concerned  with  the 
specification  of  objectives,  the  development  of  modules  in  the  socio- cultural 
core,  the  training  and  utilization  of  staff  and  the  management  of  program 
information.  If  this  project  is  to  be  implemented  as  a  regular  program 
rather  than  an  experimental  one,  additional  problems  will  undoubtedly 
arise  dealing  with  "change  in  traditional  University  procedures.  "  These 
changes  will  be  concerned  with  individualized  instruction,  degree  credit, 
traditional  grades,  final  examinations  and  other  such  features  of  "common 
college  requirements". 

Looking  briefly  at  the  problems  experienced  to  date,  the  first  major 
difficulty  experienced  was  determining  the  scope  of  the  prograuxi.  It  is 
easier  to  specify  objectives  for  job  skills  in  areas  where  there  is  little 
or  no  major  human  involvement;  that  is  in  science  areas  where  human 
relations  are  administrators'  responsibilities.  It  is  extremely  difficult 
to  identify  specific  skills  needed  by  teachers.  That  this  is  so  is  no  reason 
to  back  away  from  the  problem.  In  fact,  the  designing  of  a  program  and 
then  observing  and  evaluating  it  as  a  process  is  the  only  way  in  which 
greater  understanding  of  teaching -learning  will  come  about.  If  properly 
conceived  and  implemented,  the  performance -based  teacher  education 
movement  will,  in  the  next  few  years,  provide  unparalleled  opportunities  i 
educational  research. 

The  development  of  socio -cultural  modules  has  been  a  problem  because 
of  lack  of  understanding  of  what  is  needed  and  lack  of  agreement  about  what 
should  be  done.  In  areas  where  affective  learning,  as  opposed  to  cognitive 
and  psychomotor  learning,  is  the  major  objective  there  appears  to  be  little 
concensus  concerning  conceptual  strategies  for  implementation.  Again, 
the  key  to  greater  understanding  of  the  problems  involved  depends  on 
program  design  and  long  term  evaluation  of  consequences. 

Staff  training,  retraining  and  integration  is  typically  difficult  in 
any  new  comprehensive  and  systematic  change  in  institution  functioning. 

Such  problems  are  particularly  difficult  in  the  field  of  education  which 
has  been  observed  to  be  traditionally  conservative  and  "change  resistant". 

An  additional  factor  operating  in  opposition  to  change  which  is  somewhat 
unique  to  a  university  culture  is  the  notion  of  "academic  freedom".  It  is 
not  that  academic  freedom,  per  se,  is  an  inherent  impediment  to  progress; 
but  rather,  some  individuals  interpret  the  freedom  concept  in  ways  which 
appear  to  be  "self -protective".  Newness  can  always  be  expected  to  be 
"suspect"  and  "resisted"  so  that  careful  attention  to  in-service  training 
and  staff  involvement,  beginning  with  program  conceptualization  and  extending^ 
throughout  the  life  of  the  program,  will  more  than  justify  the  effort.  There 


is  no  substitute  for  personal  committment  to  what  one  is  doing,  and,  unless 
every  staff  member  associated  with  a  project  has  a  significant  opportunity 
to  interact  with  the  project,  there  can  be  but  slight  hope  for  the  program 
to  be  a  success. 

For  any  program  to  be  continuously  effective,  whether  the  prograun  is 
traditional  or  experimental  (or  is  known  by  any  other  name),  it  must  be 
subject  to  adjustive  control.  This  is  merely  a  way  of  saying  that  any 
system  to  continue  to  be  effective  (work)  and  to  be  increasingly  efficient 
in  achieving  any  defined  goal,  must  change.  It  is  not  easy  to  change  any  system 
in  functionally  effective  directions.  Change  in  effective  directions  requires 
definite  kinds  of  input,  for  without  input  there  can  be  no  output.  Short  of  the 
alchemists'  dreams  you  cam  never  get  out  of  amy  program  products  whose 
components  are  not  put  into  the  programn.  Some  type  of  "system's  man¬ 
agement"  must  be  applied  to  provide  feedback  (input)  into  the  system.  In 
our  program,  to  make  observations  about  module  effectiveness  as  instruc¬ 
tional  events  we  must  collect  descriptive  data  on  such  factors  as:  learner 
interaction  time  with  the  module,  the  niunber  of  posttest  attempts,  re¬ 
cycling  events  amd  elected  instructional  alternatives.  Such  descriptors 
enable  one  to  have  some  knowledge  applicable  to  second  generation  module 
design  and  the  way  the  population  of  learners  being  served  interacts  with  the 
modules.  Since  the  specific  bits  of  information  desired  are  extremely 
numerous,  it  is  only  appropriate  to  provide  technological  assistance  in  its 
management.  For  this  reason,  computer  assistance  is  a  most  likely 
choice  for  this  phase  of  management.  In  most  educational  systems,  addi¬ 
tional  data  for  administrative  reasons  is  also  desired.  For  space  assign¬ 
ment,  material  needs,  individual  progress,  grades,  degree  credit,  staff 
assignment  auid  similar  needs  a  large  amount  of  data  must  be  recorded. 

Again,  a  computer  appears  to  be  one  solution  to  the  storage  and  retrieval 
needs  of  the  program. 

Finally,  I  would  like  to  make  some  observations  about  the  performance - 
based  movement  in  the  United  States.  These  are  based  on  broader  knowledge 
than  that  provided  by  observing  the  U.  T.  El  Paso  experimental  program  by 
itself.  First,  the  state  of  Texas  now  has  legal  committments  to  require, 
within  a  five  year  transition  period  beginning  September  1,  1972,  the 
development  and  implementation  of  competency/performance-based  programs 
C/f  teacher  education  leading  to  teacher  certification.  All  new  programs 
approved  for  an  institution  (college  or  university)  after  September  1,  197Z, 
must  be  competency /performance-based  and  all  presently  approved  pro¬ 
grams  must  be  converted  to  performance  programs  by  September  1,  1977. 
Second,  this  requirement  of  the  State  of  Texas  is  not  unique  as  evidence 
of  interest  in  or  coxnmittment  to  the  competency  concept.  Utah,  Kansas, 
Nebraska,  California,  Oregon,  New  Hampshire,  New  Jersey,  Minnesota, 
Florida,  New  York  and  Washington  are  known  to  be  actively  planning  in 
this  direction.  Although  it  may  be  slightly  premature  to  predict,  it  does 


appear  to  me  that  the  performance  concept  will  either  be  required  or 
permitted  by  all  states  within  a  three  to  five  year  period.  Such  sweeping 
change  has  not  previously  characterized  teacher  training  in  this  century. 
What  the  final  outcome  of  all  this  portends  is  even  more  difficult  to  assess 
at  this  point  in  time.  However,  one  further  inescapable  observation  can 
be  made.  The  training  of  teachers  directly  relates  to  the  products  they 
produce  --  the  school  children  of  the  United  States.  It  appears  safe  to 
assume  that  performance  elements  will  be  characteristic  of  all  phases 
of  learning  within  a  short  period  of  time.  Hopefully  these  changes  will 
not  be  "change  for  change  sake"  but  also  will  produce  better  adjustments 
to  the  problems  facing  education  in  the  United  States  today. 


V-28 


DR.  JOHN  P.  FRY 


Dr.  Fry  is  currently  assigned  as  Research  Scientist,  HumRRO  Division 
No.  5,  Fort  Bliss,  Texas. 

He  received  a  BS  in  Psychology  from  Pennsylvania  State  University  and  a 
MA  in  Psychology  and  a  PhD  in  Engineering  Psychology  from  Michigan 
State  University.  Prior  to  his  present  assignment,  he  served  as  Senior 
Graduate  Research  Assistant,  Learning  Service,  Michigan  State  University. 

He  is  a  member  of  the  American  Educational  Research  Association,  the 
American  Psychological  Association,  and  the  Hiunan  Factors  Society.  He 
has  published  a  number  of  research  studies  in  professional  journals  and 
developed  workshop  materials  for  Army  officers  and  school  administrators. 


PROCEDURES  FOR  IMPLEMENTING 
SOFT-SKILL  TRAINING  IN  CONARC  SCHOOLS 

John  P,  Fry 

HumRRO,  Division  No  5,  Fort  Bliss,  Texas 


During  the  past  nine  months,  one-halt  of  Work  Unit  MODMAN  efforts  have 
been  directed  toward  investigating  procedures  for  systems  engineering 
soft- skill  job  functions  (e.  g* ,  command,  supervision,  and  leadership)  such 
as  are  fovind  in  Officer  Advanced  Courses  (C-22)  and  similar  career-type 
instructional  programs. 

As  some  of  you  may  recall,  a  two-page  outline  of  a  general  approach  toward 
soft- skill  training  was  included  in  the  questionnaire  which  we  sent  to  all 
CONARC  schools  to  help  us  define  **soft-skilis*\  Respondents  were  asked 
for  their  reaction  and  other  possible  approaches.  So,  first,  I*d  like  to 
discuss  briefly  those  reactions. 

Second,  Fm  going  to  describe  how  that  approach  fits  with  the  approach 
taken  by  the  Chemical  School  in  their  C-22  course. 

Third,  and  finally,  seven  areas  for  improvement  will  be  outlined.  In  genera 
we  feel  that  our  efforts  should  have  considerable  impact  on  soft- skill  training 
effectiveness  in  CONARC  schools.  We  invite  your  comments  and  criticisms. 

Reactions  to  a  Problem-Solving  Approach  to  Soft-Skill  Training 

Basically,  we  proposed  that  soft- skills  could  be  learned  best  by  an  instruc¬ 
tional  approach  (system)  which  was  based  primarily  on  a  problem-solving 
framework  or  structure.  Obviously,  as  change  occurs  or  as  time  passes, 
memorized  facts  have  a  very  short  life-span.  In  contrast,  solving  problems 
is  not  only  far  more  meaningful  to  the  learner,  but  it  is  also  more  transfer¬ 
able  to  future  job  assignments,  especially  where  unique  or  novel  situations 
not  now  anticipated  or  foreseen  will  be  encountered. 

The  characteristics  of  a  problem-solving  approach  include: 

1.  A  job  function  orientation.  The  focus  is  on  broad-based  job 
functions  or  skills  that  cut  across  any  job  requirement. 

2.  An  experiential  orientation.  The  active  participation  and  involve¬ 
ment  of  the  learner  in  performing  is  sought,  rather  than  just  reading  or 
listening. 
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3.  A  simulation  orientation.  Transfer  of  training  is  enhanced  by 
practice  in  problem  situations  which  simulate  those  found  in  real-life, 
on-the-job,  assignments;  problem  situations  are  complex,  ambiguous 
and  emergent. 

4.  A  cooperative -evaluation  orientation.  Opportunities  to  experiment 
with  new  techniques,  to  make  mistakes,  and  to  receive  immediate  feedback 
(primarily  from  peers)  are  emphasized.  Evaluation  is  for  feedback  pur¬ 
poses  (knowledge  of  results)  rather  than  for  competitive  (grading)  purposes. 

5.  A  fimctional  context  orientation.  Concepts  and  principles  are 
introduced  when  the  student  needs  them.  Self- instructional  materials  are 
the  primary  means  of  learning  basic  concepts  and  principles. 

6.  A  small-group  orientation.  Small  groups  or  teams  of  students  are 
formed,  each  with  a  learning  "facilitator, "  rather  than  one  lecturer  for  a 
whole  class.  Peer  instruction  is  employed  whenever  possible. 

7.  A  learning-  and  student- centered  orientation.  Since  student  learning 
is  the  goal,  students  needs  are  met  on  demand  by  immediate  administrative 
response. 

Appendix  A  includes  initial  proposal,  basic  reactions  to  it,  reservations, 
and  other  approaches  recommended  by  respondents.  In  summary,  reactions 
were  very  encouraging;  77%  of  the  respondents  actually  wrote  down  positive 
remarks  like  "Excellent! "  or  "I  like  it  very  much.  .  . .  ".  More  noteworthy 
was  the  remarkable  amount  of  agreement  among  respondents.  In  fact, 
several  schools  have  either  already  established  a  similar  instructional 
approach  or  are  planning  to  do  so. 

Reservations  to  the  approach  centered  around  two  problems:  how  to  obtain 
the  kind  of  instructors  required  and  how  to  evaluate  students  without  being 
overly  subjective.  Chaplain  O’Shea  of  the  Chaplain  School,  having  had  experi¬ 
ence  with  a  similar  approach,  provided  two  insightful  comments:  (1)  learning 
through  "discovery"  while  in  the  process  of  "doing"  something,  and  (2)  intro¬ 
ducing  cognitive  information  at  the  point  of  need,  so  that  it  is  most  useful  to 
the  student  (functional  context). 

Other  approaches  recommended  by  respondents  included  specific  methods 
such  as  In-Basket,  Case  Studies,  Micro-Teaching  or  having  students  attempt 
problem-solving  without  adequate  resources  or  time.  Remaining  suggestions 
were  quite  heterogenous,  but  all  could  fit  within  a  basic  framework  of 
problem-solving. 


In  conclusion,  there  seems  to  be  much  support  CONARC  systems 

engineers  for  a  problem-solving  approach  to  the  soft-skill  training.  Let 
us  look  now  at  the  Chemical  School’s  implementation  of  such  an  approach. 

The  Chemical  School's  Problem-Solving  Approach 

In  early  June,  I  visited  the  Chemical  School  at  the  invitation  of  Mr.  Thomas 
E.  Chandler,  Educational  Advisor^  and  LTC  Thomas  R.  Roark,  Advance 
Course  Director,  to  evaluate  their  re-designed  C-22  course. 

The  Chemical  Officer  Advanced  Course  had  had  a  general  history  of  dis¬ 
satisfied  students.  That  is,  they  were  either  unmotivated,  complained  all 
the  time,  or  dared  instructors  to  teach  them.  This  is  typical  behavior,  I 
understand,  of  Basic  and  Advanced  Course  students. 

In  the  old  system,  there  was  little  coordination  among  instructors  or 
departments.  As  a  consequence,  each  instructor  hung  his  piece  of 
"important"  knowledge  on  students  as  they  passed  that  instructor's  station- - 
exactly  like  an  assembly  line.  Further,  instructors  were  concerned  more 
with  how  to  lecture  than  with  how  students  learn  or  with  how  they  will  use 
information  once  they  take  up  job  assignments. 

By  backing  off  far  enough  from  the  compartmentalized  structure  of  the 
original  course,  LTC  Roark  and  his  associates  were  able  to  make  use  of  the 
systems  engineering  principle  of  job  analysis.  That  is,  they  identified  that 
the  graduates  basically  ought  to  be  able  to  solve  problems  and  make  decisions. 
Furthermore,  from  now  on,  due  to  the  variety  of  job  assigximents,  only  five 
to  ten  per  cent  (5-10%)  of  the  graduates'  time  would  be  spent  functioning  as 
chemical- oriented  branch  officers.  Thus,  they  would  soon  be  required  to 
solve  a  variety  of  problems,  most  of  which  would  be  unrelated  to  their  pre¬ 
vious  "hard- skill"  experience  and  training.  (NOTE:  Chemical  branch 
officers  seldom  ever  become  connmanders,  mainly  they  serve  as  staff  officers.) 

In  fact,  when  incumbent  commanders  were  asked  what  abilities  they  most 
desired  in  Chemical  Corps  officers,  problem-solving  ability  ranked  right 
behind  proficiency  in  CBR.  Other  broad-based  job  functions  were  also 
identified,  such  as  interpersonal  and  communication  skills.  A  Graduate 
Profile  was  derived  (Appendix  B)  which  was  quite  general  but  met  all  identi¬ 
fied  on-the-job  needs.  When  more  specific  training  objectives  were  proposed, 
only  arguments  ensued.  It  was  clear  to  them  that  task  inventories,  as  advo¬ 
cated  by  conventional  systems  engineering  procedures,  were  trivial  and 
unsuited  for  the  "soft- skill"  training  required  by  Advanced  Course  students. 

Thus,  LTC  Roark's  primary  training  goal  became  devising  an  environment 
where  students  could  solve  problems  within  the  context  of  the  jobs  these 
officers  might  encounter.  ( 


Problem-Solving  As  A  Framework 


Most  of  you  are  aware  that  in  most  educational  settings^  if  problem-solving 
is  required  of  students,  it  is  quite  specific*  That  is,  problems  and  methods 
of  solution  are  well  specified  and  the  student  is  evaluated  on  his  arriving  or 
not  arriving  at  a  correct  solution--  not  on  his  ability  to  identify  the  problem, 
to  select  the  best  method  of  analysis,  or  to  analyze  the  problem  and  develop 
possible  solutions.  Likewise,  under  conventional  methods  of  instruction, 
the  learner  leaves  a  course  mostly  with  answers  to  "old"  or  routine  problems, 
not  necessarily  with  skills  in  how  to  solve  new  or  emergent  ones. 

However,  if  students  are  required,  as  they  were  here,  to  seek  out  and  apply 
information  in  solving  non- routine  problems  that  closely  approximate 
(simulate)  typical  officer  on-the-job  performance  requirements,  it  follows 
that  not  only  will  they  be  able  to  begin  functioning  effectively  once  they  face 
on-the-job  problems,  they  will  also  know  how  to  seek  out  knowledge  from 
whatever  resources  are  available- -be  they  TMs,  ARs,  SOPs,  peers,  staff 
members,  subordinates,  or  consultants*  Thus,  they  become  skilled  at 
locating  information  and  using  it,  rather  than  just  becoming  repositories 
of  rapidly  aging  knowledge.  However,  as  far  as  knowledge  acquisition  goes, 
they  don't  learn  all  of  the  information  normally  spewed  out  in  lectures; 
rather,  they  learn,  through  inquiry  and  initiative,  to  find  samples  of  that 
whole- -that  which  is  relevant  and  meaningful  toward  solving  realistic 
problems.  It  should  be  noted  that  such  a  problem-solving  framework  or 
"hollow  shell"  easily  accommodates  all  subject  matter  presently  perceived 
as  essential  to  graduates,  or  any  new  information  which  may  become  required 
learning  in  the  future. 

The  Chemical  School's  Advanced  Course  36  weeks  of  training  is  roughly  split 
into  six  phases  or  six  levels  of  coxnmand  (each  can  be  conceptualized  as  a 
learning  vehicle): 

Phase  1-- Company /Battalion 
Phase  II- -Brigade 

Phase  in- -Radiological  Safety  -  Prefix  5 

Phase  IV- -Division 

Phase  V- -Installation  (CONUS  Base) 

Phase  VI- -Higher  HQ/Logistics 

(NOTE:  Phase  III,  Radiological  Safety,  is  taught  by  conventional  instruc¬ 
tional  techniques,  as  it  is  quite  quantitative /technical  and  apparently  fits  that 
instructional  methodology  quite  well. ) 

The  reason  for  six  phases  is  functional.  As  graduates  progress  upward  in 
organizational  levels  of  the  Army,  they  will  be  asked  to  solve  similar 
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problems  (e.  g. ,  drug  abuse),  but  at  d'  -ferent  l^veis  of  complexity.  In 
other  words,  during  the  36  weeks  (exc^^pt  during  Phase  III)  students  might 
encounter  the  same  problem  five  times,  but  at  increasingly  more  difficult 
and  ^.lore  complex  levels. 

Problem  situations  were  derived  from  many  sources,  but  the  emphasis  is 
on  ^'realistic*'  situations  where  problems  are  imbedded  in  the  information 
supplied.  A  local  chemical  battalion  contributed  many  problem  situations 
which  fit  an  initial  ”in-basket‘*  type  instructional  methodolo^.  In  later 
phases  of  the  course,  problems  were  introduced  via  operations  orders. 
There  were  no  school  solutions  and  often  no  final  solution  was  required. 
Students  usually  were  required  to  brief  a  field  grade  officer  familiar  with 
the  problem  area  before  complete  analysis  could  be  carried  out. 

Small  Groups  and  Evaluation 


To  implement  the  problem-  solving  approach,  small  groups  of  four  to  six 
students  became  the  basic  instructional  unit.  A  small  group  format  provides 
students  with  opportunities  to  help  each  other  learn.  At  this  stage  of  pro¬ 
fessional  development,  it's  not  too  uncommon  for  a  given  student  to  be  more 
knowledgeable  and  more  experienced  about  a  given  topic  than  even  instructors. 
Thus,  evaluation,  in  the  form  of  immediate  feedback,  can  be  meaningfully  ^ 
given  by  one  student  to  another  student.  In  fact,  students  are  often  more 
willing  to  accept  and  to  learn  from  their  mistakes  if  evaluation  is  in  the  form 
of  immediate  feedback  from  peers.  Moreover,  learning  from  mistakes  is 
enhanced  and  small  groups  can  become  teams  in  which  cooperative  learning 
takes  precedence  over  competitive  learning. 

Nonetheless,  students  were  critiqued  daily  by  consultants  and  "rented  faculty" 
who  evaluated  their  products  (analyses  and  briefings).  Criteria  were  no 
longer  based  on  how  many  facts  students  knew,  but  could=they  use  factual 
information.  The  level  of  criteria  became  analogous  to  that  of  the  real 
world,  where  a  general  evaluation  is  more  relevant  to  success  or  failure. 

In  any  event,  except  for  CONARC  requirements  for  class  standing  (top  20%), 
competitive  grading  was  eliminated^ 

Student- Centered  Learning 

Instead  of  platform  lecturers,  instructors  become  resource  personnel  capable 
of  consulting  with  student  groups  and  advising  them  in  their  solving  assigned 
problems.  The  emphasis  was  on  learning  and  student  control  of  that  learning. 


Each  of  the  six  phases  is  coordinated  by  a  Phase  Leader,  who  is  assisted  by 
faculty  consultants- -experts  in  that  particular  phase.  Consultants  were  only 
required  to  be  available  for  two  hours  per  day  during  their  five  to  six  week 


tour  of  duty.  Since  they  served  only  as  resources  (they  had  no  lesson  plans, 
films,  or  slides  and  gave  no  lectures),  student  teams  met  with  them  during 
the  two  hours  as  they  felt  it  necessary.  Thus,  in  a  typical  40-hour  week, 
there  were  only  eight  (8)  hours  that  required  student  attendance  (e«  g. ,  guest 
speakers,  seminars,  or  electives).  Although  students  were  able  to  come 
and  go  as  they  pleased,  in  practice,  assigned  problems  caused  them  to  work 
many  more  than  40  hours  per  week. 

Summary  Thus  Far 

So  far  we've  discussed  an  instructional  approach  to  soft- skill  training  which 
is  problem-solving  based.  It  indudes  expe riential  learning  in  small  groups. 
Students  solve  simulated^  job  related,  problems  by  means  of  broad-based 
job  functions,  acquiring  information  in  a  functional  context.  Students  are  in 
control  and  are  evaluated  non- competitively.  Thus,  all  seven  characteristics 
outlined  previously  are  utilized  in  the  Chemical  School's  C-22  Course. 

Seven  Areas  For  Improvement 

The  first  class  to  learn  under  the  new  system  was  about  three  weeks  from 
graduation  when  I  interviewed  one  team  of  six  students.  In  general,  they 
considered  their  training  to  be  worthwhile  because  they  were  learning  infor¬ 
mation  through  solving  realistic  military  problems.  They  acknowledged  that 
their  motivation  to  continue  learning,  even  during  the  final  weeks  of  the  nine- 
month  course,  resulted  from:  (1)  peer  pressure,  (2)  a  past  history  of 
satisfaction  from  having  achieved  meaningful  learning,  and  (3)  the  challenge 
of  the  present  problems. 

Nonetheless,  they  felt  several  areas  needed  improvement.  Likewise,  our 
experience  here  at  Fort  Bliss  in  developing  and  testing  out  an  instructional 
system  very  similar  to  that  of  the  Chemical  School,  but  much  shorter, 
suggest  that  a  number  of  improvements  can  be  made  without  destroying  its 
integrity  or  viability. 


While  working  on  MODMAN  concepts  and  procedures,  we  have  been  fortunate 
to  have  been  able  to  try  them  out  in  a  Performance  Counseling  Workshop, 
which  we  had  previously  developed  for  MG  R.  L«  Shoemaker  and  the  Fort 
Bliss  Modem  Volimteer  Army  (MV A)  office.  As  it  turned  out,  the  workshop 
gave  us  plenty  of  time  to  repeatedly  test  out  our  new  methodology,  evaluate 
its  "bugs",  revise  it,  and  then  re-test  it.  Thus,  our  final  produce  incor¬ 
porates  a  number  of  methodological  innovations  which  should  help  to  improve 
the  basic  characteristics  of  the  Chemical  School  Advanced  Course. 


The  Performance  Counseling  Worksho 


In  general,  our  workshop  is  problem-solving  based.  In  fact,  students  learn 
to  be.  'yme  facilitators  of  both  individual  and  group  problem-solving.  Like¬ 
wise,  it  includes  the  seven  characteristics  mentioned  previously:  broad-based 
job  functions  are  learned  experientially  via  solving  simulated  problems  in 
small  group  settings  while  information  is  acquired  in  a  functional  context 
under  student- centered  conditions  a  xd  where  immediate  evaluation  from  peers 
is  emphasized. 

The  basic  instructional  design  can  be  conveniently  viewed  as  a  sequence  of 
five  training  methods  which  approximate  more  and  more  closely  real-life 
situations.  Table  1  lists  these  five  training  methods,  each  of  which  reflects 
a  degree  of  approximation  to  on-the-job  performance,  along  a  ‘’fidelity  of 
simulation"  scale.  Associated  student  behaviors  and  training  objectives,  as 
well  as  the  training  goal  (behavior  application  on-the-job)  are  also  listed  on 
the  table.  (NOTE:  In  a  course  (e.  g.  ,  C-22),  this  sequence  of  training 
methods  would  be  repeated  as  many  times  as  necessary. ) 

T  raining  Methodology 

The  first  training  method,  introductory  briefing,  is  used  only  for  student  ^ 
motivation  and  overview  purposes.  Nonetheless,  feedback  from  students  is 
sought  so  as  to  determine  students*  entry  level  and  to  detect  misperceptions 
of  training  intent. 

The  second  training  method  involves  self-paced  learning  of  concepts  and 
principles  by  means  of  self-instructional,  modularized  materials  where 
actions,  conditions,  and  standards  are  specified.  As  soon  as  a  student  feels 
that  he  has  learned  the  objectives  for  each  module  of  the  material,  he  can 
test  himself  (Method  3).  Nonetheless,  such  cognitive /verbal  learning  is 
integrated  with  skill  acquisition  as  much  as  possible,  thereby  making  use  of 
the  ’’functional  context"  principle  of  training.  Also,  peer- instruction  is 
emphasized  wherever  appropriate. 

The  last  two  training  methods  require  that  small  groups  of  students  be  formed, 
each  with  an  instructor  or  learning  facilitator.  His  function  is  to  serve  as  a 
learning  resource  to  the  group,  to  insure  quality  control  and  to  guarantee  a 
risk-free  environment.  Class  time  is  devoted  to  either  observation  of 
behavior  or  repeated  practice  of  analytical  or  behavioral  skills  in  simulated 
problem  situations.  Thus,  all  students  in  each  group  either  observe  or  take 
part  as  participants.  Observers  are  at  least  able  to  react  to  the  problem 
situations  vicariously,  enabling  them  to  participate  constructively  in  debrief¬ 
ings  that  follow.  ^ 
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In  general,  initial  simulation  exercises  are  very  short  and  simple,  with 
clear*cut  solutions,  requiring  direct  application  of  only  a  few  skills 
(similar  to  micro-teaching).  Later,  as  students  become  more  confident 
and  more  skilled,  the  level  of  fidelity  of  the  simulated  problem  situations 
is  correspondingly  increased  in  complexity  and  difficulty.  The  goal  is  to 
maximize  transfer  from  the  training  environment  to  the  job  environment. 

A  Basic  Difference 

In  the  Chemical  School  Advanced  Course  the  emphasis  is  on  obtaining  depth 
of  content  by  concentrating  on  acquiring  information  through  the  development 
of  solutions  to  simulated  problems.  Although  students  try  to  find  out  how  to  best 
use  information,  analyzing  and  identifying  strategies,  processes,  or  methods 
of  attack  are  not  the  focus  of  learning.  Effective  processes  are  supposed  to 
emerge,  somehow,  from  the  act  of  problem- solving--!,  e. ,  trial  and  error 
learning.  Of  course,  learning  from  peers,  using  past  experience  or  model¬ 
ing  on  consultants  are  typical  trial  and  error  methods  of  acquiring  processes. 

In  contrast,  both  we  and  the  Chaplain  School  focus  on  obtaining  depth  of  con¬ 
tent  by  concentrating  on  processes- -the  how  of  getting  things  done.  For 
example,  both  of  us  would  be  interested  in  how  to  lead  small  group  discussion 
or  how  to  handle  or  counsel  an  over -emotional  soldier.  In  addition  to 
advancing  specific  process  techniques,  we  both  state  behavioral  objectives 
at  the  outset,  (NOTE:  In  the  Chaplain  School  Advanced  Course  students  are 
given  much  more  freedom  to  modify  objectives  and  MOI  than  in  our  workshop. 
However,  they  are  less  problem-solving  oriented,  since  command  or  leader¬ 
ship  positions  are  not  of  Interest  to  them. ) 

Recommended  Improvements 

What  follows  are  seven  recommended  improvements  to  the  basic  character¬ 
istics  of  the  Chemical  School^s  C-22  Course.  Some  of  them  also  apply  to  the 
C-22  Course  at  the  Chaplain  School. 

1.  Behavioral  Objectives  Should  Be  Made  Known.  Students  should  know 
what  behaviors  they  ought  to  perform,  what  conditions  will  be  present,  and 
what  typical  performance  criteria  or  standards  are.  The  Chemical  School 
Course  designers,  in  an  effort  to  create  a  student- oriented  instmictional 
environment,  purposively  refrained  from  prescribing  what  was  to  be  learned 
because  they  felt  students  should  ’’discover''  processes  or  techniques  for 
themselves. 


This  approach  led  to  confusion  and  anxiety,  forcing  students  to  band  together 
to  try  to  discover  what  the  consultants  really  thought  was  a  good  analysis,  etc. 


Although  such  ambiguity  is  representative  of  life  and  promotes  peer 

cohesion^  it  makes  learning  difficult.  Researchers  who  have  studies 
"discovery"  as  a  learning  technique  are  convinced  that  for  exploration  to 
be  n  saningful,  the  objective  of  the  task  must  be  known  and  the  learner 
should  know  where  he  stands  with  respect  to  it  (Bruner,  1968).  In  fact, 
tasks  that  are  too  uncertain,  arouse  so  much  confusion  and  anxiety  that 
exploration  behavior  is  reduced. 

Previously,  it  was  mentioned  that  a  task  analysis  of  Advanced  Course 
content  led  to  trivial  entries.  This  does  happen  if  the  focus  is  on  content 
alone.  But  a  behavioral  analysis  of  processes  that  make  up  any  larger  or 
broad-based  job  function  can  produce  a  hierarchical  sequence  of  specific 
skills  that  are  not  trivial  (Whitmore,  in  press).  Thus,  the  systems  engi¬ 
neering  steps  are  valid  for  soft-skills.  All  that  is  needed  is  a  process 
point  of  view  rather  than  content. 

2.  A  •’Guided"  Discovery  Instructional  Format  Ought  To  Be  Adopted. 
"Guided"  Discovery  allows: 

(a)  Efficient  acquisition  of  concepts  and  principles  novel  to  the 
student.  He  does  not  have  to  waste  time  and  energy  reinventing  the  wheel. 

(b)  The  student  to  test  out  novel  concepts  and  behaviors  so  that  he 
can  convince  himself  that  they  work.  This  is  especially  useful  when  behav¬ 
iors  contrary  to  his  experience  and  background  are  ones  he  ought  to  be 
using. 

"Guided"  Discovery  begins  with  a  general  specification  of  behavioral  objec¬ 
tives  and  the  learning  of  basic  cognitive  concepts  and  skills.  Then,  as  the 
student  makes  mistakes  in  applying  these  concepts  or  skills  as  behaviors, 
additional  information  is  brought  in  as  needed.  Thus,  the  functional  context 
principle  of  meeting  students  needs  when  they  have  needs  can  be  utilized. 
Further,  "guided"  discovery  lends  itself  to  a  student-centered  learning 
orientation. 

Mager  and  McCann  (1961)  tried  a  similar  instructional  format  with  newly 
graduated  and  newly  hired  engineers  who  were  required  to  learn  essential 
information  about  their  new  organization.  With  only  behaviorally- stated 
training  objectives  and  a  list  of  resources  available,  a  group  of  six  "trainees 
was  able  to  complete  their  training  in  one-third  (1/3)  the  time  of  previous 
"trainee"  instruction.  Furthermore,  they  tJisplaytd  more  confidence  and 
were  able  to  work  with  less  direct  Mipervision  than  previous  "trainees". 

It  would  be  an  ideal  learning  environment  if  a  student  could  control  the  rate 
of  his  information  acquisition,  structure  the  information  to  fit  his  particular 
background,  and  obtain  additional  information  on  demand*  An  experiment 


to  test  such  an  instructional  design  proved  that  learning  is  increased,  but 
only  if  the  student  has  had  past  experience  and  success  in  making  inquiry 
and  solving  problems  on  his  own  (Fry,  1972).  Until  computers  are  built 
which  can  respond  as  described,  however,  requirements  for  efficient, 
complete  or  effective  student- control  or  "discovery"  cannot  be  met,  even 
for  information  acquisition.  Further,  since  soft- skills  involve  processes 
or  behavioral  skills  which  require  application  within  group  settings,  group 
feedback  or  "guidance"  remains  an  essential  ingredient  in  the  instructional 
design. 


3.  Acquisition  of  Information  Needs  To  Be  Made  Easier.  Several  stu¬ 
dents  saw  the  present  information  acquisition  system  as  inefficient  in  terms 
of  effort  and  time.  (NOTE:  Army  manuals  were  judged  difficult  to  read, ) 

Thus,  there  is  a  need  to  develop  short,  on-the- shelf,  learning  materials, 
preferably  self- instructional  in  design,  so  that  students  who  are  weak  in 
certain  areas  can  pick  up  necessary  skills  and  techniques  when  they  need 
them- -quickly,  efficiently,  and  on  their  own.  Such  modular- packaged  pro¬ 
grams  should  include  specific  behavioral  objectives  and  performance  criteria. 
The  learning  center  concept  is  appropriate  here.  Since  they  would  be  in 
support  of  primary  learning  goals,  their  number  and  content  should  be  based 
strictly  on  student  needs. 

Nonetheless,  peer  instruction  remains  a  valuable  means  of  utilizing  the 
resources  available  within  each  group  of  students.  Only  when  students 
exhaust  their  own  resources,  should  self- instructional  materials  be  pre¬ 
pared  or  used.  This,  of  course,  requires  repeated  faculty  testing,  sensi¬ 
tivity,  and  responsiveness  to  student  needs  instead  of  what  is  convenient  for 
administrative  purposes. 

The  Chemical  School  faculty  responded  to  their  student's  needs  by  designing 
realistic  and  relevant  problem  situations.  Now  they  need  to  listen  for  what 
is  needed  to  make  learning  more  efficient  and  at  the  same  time  allow  the 
student  to  learn  skills,  processes,  or  techniques  which  are  superior  to  those 
which  they  bring  to  school  or  to  those  known  by  their  instructors. 

4.  Additional  Process-Based  Job  Functions  Need  To  Be  Integrated  Into 
Existing  Training  Goals.  For  example,  behavioral  process  skills  for  dealing 
with  subordinates  are  beyond  the  scope  of  the  course  as  it  presently  exists. 
Other  schools,  especially,  need  to  train  their  students  in  such  areas  as  per¬ 
formance  counseling,  motivation,  and  leadership. 

A  problem-solving  instructional  framework  is  especially  suited  to  leadership 
training  because  effective  leadership/management  is  not  only  using  informa¬ 
tion;  it  is  applying  analytical  and  behavioral  skills,  appropriate  to  the  situation, 
after  an  analysis  has  been  made.  Thus,  listening  to  lectures,  reading  books. 
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or  solving  paper  problems  with  the  goal  of  urirlor :  anding  and  accepting  new 
ways  of  handling  ’’people"  problems  is  not  sufficient.  Moreover,  such  inade* 
quate  training  invariably  causes  those  who  attempt  to  try  new  approaches  of 
lead '^r ship /management  to  experience  failure  and  frustration.  As  a  conse> 
quence,  they  usually  revert  to  more  familiar  methods  when  training  concludes. 

In  general,  students  need  more  than  mere  exposure  to  alternative  behavioral 
skills;  they  need  to  practice  using  them  until  they  can  operate  outside  of 
narrow  stereotyped  frameworks. 

5.  Behavioral  Scientists  Need  To  Be  Brought  Into  the  Analysis  of  Job 
Fimctions.  Recently,  through  much  research  and  development,  behavioral 
scientists  in  industrial  and  organizational  settings  have  discovered  and 
derived  new  processes  and  behavioral  techniques  which  have  great  usefulness 
for  Army  leaders.  Are  we  going  to  have  students  learn  how  to  use  the  best 
known  state-of-the-art  techniques  or  are  we  expecting  them  to  not  only  invent 
the  wheel  but  also  discover  the  latest  and  best  tire  to  go  on  it? 

If  CONARC  schools  do  not  make  use  of  behavioral  scientists,  too  much  value 
is  placed  on  individual  instructors’  past  personal  experiences  or  their  hypo¬ 
theses  of  how  best  to  cope  with  problems  they  haven't  experienced.  In  our 
experience  the  best  available  behavioral  techniques  rarely  are  used  by  job 
incumbents,  let  alone  by  instructors.  Further,  if  instructors  (consultants) 
do  not  exhibit  the  best  known  behavioral  skills,  they  not  only  serve  as  poor 
models,  they  may  reinforce  counter-productive  behaviors  as  well. 

It  follows  that  behavioral  skills,  processes,  or  job  functions  which  lie  outside 
the  range  of  existing  faculty  experience  or  training  require  expert  analysis. 
Behavioral  scientists  appear  to  be  best  able  to  analyse  officers’  job  require¬ 
ments  to  identify  problem  areas  and  then  determine  appropriate  job-tested 
behavioral  techniques  which  will  enable  graduates  to  effectively  handle  them. 

6.  Students  and  Instructors  Need  To  Be  Trained  To  Use  a  Problem- 
Solving  Framework.  Chemical  School  students  were  confronted  with  a 
learning  environment  that  required  skills  unlike  those  useful  in  passing 
college  courses.  Thus,  "shock"  occurred  for  some.  It  took  time  and  under¬ 
standing  for  some  students  to  learn  what  to  expect  and  what  was  expected  of 
them.  The  excessive  degree  of  "freedom"  and  ambiguity  inherent  in  problem¬ 
solving  caused  some  students,  who  had  been  successful  in  "parroting  back" 
information  in  the  past,  now  to  "fail". 

Specification  of  behavioral  objectives  will  greatly  reduce  ambiguity,  but  the 
requirement  to  take  initiative  in  solving  problems  where  uncertainty  exists, 
demands  special,  systematic,  training  to  overcome  prior  conditioning 
(Campbell,  1964;  Dunbar  and  Dutton,  1972;  Fry,  1972).  Likewise,  instructors 

( 


V-40 


need  training  for  conversion  from  traditional  roles  into  consultants  or 
facilitators  of  learning.  The  Chaplain  School  C-22  Course,  in  fact,  begins 
with  a  special  five-day  workshop  to  develop  students  and  faculty  into  effec¬ 
tive  learning  teams. 

7.  Administrators  Need  Training  In  Organizational  Development  (OD) 
Skills.  In  the  beginning,  LTC  Roark  and  his  associates  re-designed  the 
C-22  Course  their  way.  But  they  had  difficulty  in  selling  it  to  faculty;  in 
fact,  they  were  lucky  to  have  been  able  to  overcome  resistance  to  their 
changes.  If  other  schools  wish  to  implement  similar  changes,  curriculum 
designers  and  administrators  need  to  know  how  to  initiate  change  and  how 
to  develop  and  adapt  their  organization  to  the  needs  of  students  and  the 
changing  environment.  Organizational  development  is  the  name  for  total- 
system,  planned -change,  processes  that  increase  organizational  effective¬ 
ness  to  the  point  where  self- renewing  practices  become  SOP  (Beckhard, 

1969;  Gardner,  1964;  Schein,  1972). 

Thus,  organizational  development  techniques,  which  include  problem¬ 
solving  skills,  not  only  aid  in  initiating  change  and  setting  goals,  they  also 
aid  in  maintaining  an  instructional  system  that  is  responsive  to  both  the 
real  world  and  students'  needs.  Although  systems  engineering  of  training 
procedures  are  supposed  to  accomplish  similar  goals,  nowhere  in  the  regu¬ 
lations  are  mention  of  the  necessary  processes.  That  the  Chemical  School 
designers  have  already  \indertaken  steps  to  avoid  hardening  of  the  arteries 
or  inbreeding  of  ideas  shows  the  need  for  self- renewal.  Encouraging  students 
to  challenge  the  instructional  methods,  content,  and  goals  or  rotating  staff 
personnel  has  become  their  main  means  of  self- renewal.  Nonetheless,  other 
methods  are  possible  and  probably  will  be  needed  as  time  will  bring  new 
administrators  and  others  who  will  challenge  the  existing  system.  Reactionary 
change  could  replace  planned  change  unless  the  benefits  of  a  problem-solving 
format  can  be  experienced  by  newcomers  within  a  management  system  which 
is  itself  problem-solving  and  continually  self- renewing. 

CONCLUSION 

We  have  outlined  an  approach  toward  implementing  soft- skill  training  in 
CONARC  schools,  specifically  C-22  courses.  At  this  point,  the  instructional 
model  is  fairly  well  conceptualized  and  a  testing  out  phase  has  begun.  Further 
development  and  testing  should  soon  provide  sufficient  experience  and  feed¬ 
back  to  generate  specific  procedures  and  processes  which  will  enable  CONARC 
schools  to  adopt  and  implement  a  similar  instructional  format.  Constraints 
imposed  by  practical  reality  will  no  doubt  play  a  part  in  the  final  design  at 
each  school. 


% 


V-41 


REFERENCES 


Beckhardt  Richard.  Organization  Development:  Strategies  and  Hodela.  ^ 

Hanlo  Park,  California:  Addison-Wesley  Publishing  Company,  1969.  | 

Bruner,  J.  S.  "The  Act  of  Discovery,"  Harvard  Educational  Retfieu^  I 

31,  1961,  21-32.  j 

Caispbell,  V.  N.  "Self-Direction  and  Programmed  Instruction  for  Five  | 

Different  Types  of  Learning  Objectives,"  Payohology  in  the  Schools ,  ] 

1,  1964,  348-359. 

Dunbar,  Roger  L.  M.  and  Dutton,  John  M.  "Student  Learning  in  a 

Restructured  Environment,"  Journal  of  Research  and  Development 
in  Education^  6(1) ,  1972,  26-37. 

Fry,  John  P.  "The  Interactive  Relationship  Between  Inquisitiveness 
and  Student  Control  of  Instruction,"  Journal  of  Educational 
Psychology t  63,  1972.  In  Press. 

Gardner,  John  W.  Self~-Renewal:  The  Individual  and  the  Innovative 
Society,  New  York:  Harper  Colophon  Books,  1964. 

Hager,  R.  F.  and  McCann,  J.  Learner-Controlled  Instruction.  Palo 
Alto,  California:  Varlan  Association,  1961. 

Scheln,  Edgar  H.  Process  Consultation:  Its  Role  in  Organization 

Development.  California:  Addison-Wesley  Publishing  Company,  1969. 

Whitmore,  Paul  G.  The  Concept  of  the  Job  Model:  A  Partial  Job 
Model  for  Army  Officers,  HumRRO  Technical  Report.  In  Press. 

I 


V-42 


APPENDIX  A 


A  Look  Into  the  Future 

Now  that  you've  had  a  chance  to  help  u«t  define  what  "aoft-ekitV 
Jobs  are  or  are  not,  we'd  like  your  reaction  to  our  general  conception 
of  how  they  should  be  taught. 

Basically,  we  feel  that  acquisition  of  most  of  the  knowledges  and 
skills  of  "aoft-akill"  courses  must  Involve  the  active  participation 
and  involvement  of  the  learner.  Furthermore,  for  such  knowledges  or 
skills  to  be  transferable,  they  must  be  used  by  the  learner  In  the 
active  solving  of  training  problems  very  similar  to  those  found  on- 
the-job  In  real-life  military  assignments.  As  such,  the  Instructor  Is 
not  concerned  with  providing  answers  or  solutions  to  the  ler-ner  but 
rather  with  providing  him  with  a  variety  of  simulated  "emergent'’  situ¬ 
ations.  There,  the  learner  can  experiment  with  new  techniques,  make 
mistakes,  and  learn  from  them  (receive  Immediate  feedback)  without 
fear  of  embarrassment  or  fear  of  evaluation. 

Furthermore,  we  feel  that  basic  concepts  and  principles  can  be 
learned  effectively  by  Individualized,  self-paced  methods.  Since  learn¬ 
ing  objectives  (Including  actions,  conditions  and  criteria)  will  be 
stated  In  behavioral  terms,  the  learner  will  know  when  he  has  suffi¬ 
ciently  mastered  .hem.  Thus,  before  he  attempts  to  apply  them  to 
simulated  situations,  he  will  be  required  to  be  at  least  cognitively 
knowledgeable  In  the  basics. 

Next,  small  groups  of  learners  are  formed,  each  with  an  instructor 
or  learning  facilitator.  His  job  function  will  be  to  provide  essential 
resources  and  guarantee  a  non-threatening  environment,  one  removed  from 
disruption  and  undue  observation.  Class  time  Is  devoted  to  repeated 
practice  of  knowledges  and  skills  In  simulated  situations.  Peer  evalu 
atlon  Is  the  primary  means  of  feedback  and  Is  Immediate,  accurate,  and 
corrective  In  nature. 

For  exaiq>le,  assume  you  were  about  to  design  a  POI  to  teach  combined 
arms  doctrine  In  the  Advanced  Course  (say  Air  Defense  doctrine  to  Infantry 
types) .  Why  not  have  students  learn  the  Information  from  the  point  of 
view  (perspective)  of  one  who  has  just  taken  command  of  an  appropriate 
unit  where  the  knowledges  and  skills  of  Air  Defense  doctrine  are  needed 
to  solve  a  military  problem?  Thus,  Instead  of  having  to  memorize  a  list 
of  concepts,  the  student  would  only  be  required  to  know  where  they  are 
located  (perhaps  best  In  a  condensed  version  of  FMs)  and  be  able  to  use 
them  In  solving  realistic  problems. 

Obviously,  as  chsnge  occurs  or  as  time  passes,  memorized  facts  have 
a  very  short  life  spsn — but  having  to  solve  problems  In  school  by  use  of 
concepts  or  principles  Is  not  only  far  more  meaningful  to  the  learner  but 
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also  nore  transferable  to  future  Job  assignments,  especially  where  the 
officer  will  encounter  unique  or  novel  sltuatlcns  net  now  anticipated. 


Of  course »  such  terminal  tasks  car  be  very  complex  and  difficult. 
Therefore,  a  hierarchically  arranged  series  of  less  difficult  learning 
ste  >8  are  required  before  final  terminal  objectives  are  attempted.  In 
addition,  modification  and  updating;'  of  simulation  situations  to  reflect 
change  would  be  built  Into  the  instructional  design. 

Before  reacting  to  these  Ideas,  please  read  the  following  definition 
of  simulation. 

Simulation 

simulation  la  a  technique  for  modeling  real-life  complex  situations. 

By  the  actual  practice  of  new  inodes  of  responding  to  stimuli  which  are 
highly  similar  to  real-life  situations,  simulation  exercises  are  most 
effective  In  extinguishing  old,  inappropriate  responses  and  establishing 
new,  appropriate  responses.  This  is  especially  true  when  "knowing  what 
to  do,  and  how  to  do"  Is  not  equivalent  to  "being  able  to  do". 

To  Insure  maximum  transfer  of  learned  skills  to  real-life  situations, 
training  situations  should  be  complex,  ambiguous,  and  emergent^  so  that, 
as  In  real  life,  they  require  the  trainee  to  sort  out  the  Important  facts 
from  the  confusion  of  events  surrounding  him  and  to  identify  the  essence  of 
a  problem  as  it  arises.  For  example,  one  of  the  more  Important  skills  a 
leader  must  develop  In  order  to  be  effective  Is  to  be  able  to  recognize 
when  a  problem  exists  and  to  be  able  to  identify  it  correctly. 

Furthermore,  simulation  enables  the  learner  to  practice  In  a  large 
variety  of  critical  situations  in  a  short  time.  Thus,  he  can  reach 
established  criteria  relatively  quickly.  In  addition,  simulation  pro¬ 
duces  considerable  Involvement  and  motivation  on  the  part  of  participants; 
a  condition  considered  necessary  for  higher  level  type  learning  to  occur. 
Simulation  is  also  well  suited  to  role-playing,  another  method  of  obtain¬ 
ing  learner  Involvement  and  concern.  Finally,  self-evaluation  by  trainees 
of  their  performance  can  be  easily  encouraged,  especially  through  compari¬ 
son  with  responses  of  other  participants. 

In  general,  initial  simulation  exercises  should  be  simple,  l.e.,  with 
clear-cut  solutions  and  permitting  direct  application  of  concepts  and 
principles.  Later,  as  success  builds  confidence  and  skills  Increase, 

■ore  difficult  simulation  exercises  can  be  tried.  When  to  Introduce  them 
Is  subjective,  it  depends  on  student  learning  rate  and  training  time 
limitations.  The  goal  is  to  facilitate  the  transfer  of  learned  principles 
and  skills  to  real-life  problem  situations  with  minimum  effort  and  maximum 
confidence. 
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BmIc  raactlons  by  35  respondents  to  the  question:  Uhat  do  you  think 
about  tJw  above  approach  to  training  in  the  eoft-ekill  area? 

•  Sound  approach....  Ua  are  forecasting  essentially  the  sane 
approach... FY  73.  Agree  with  the  approach....  We  hope 
to  Include  much  more  individual  and  sautll  group  work  In  our 
advanced  course.  This  approach  Is  presently  being  tested 
on  a  trial  basis....  [Six  respondents  Indicated  this 
approach  or  similar  approach  Is  presently  being  used  or  planned.] 

•  valuable 

•  definite  application 

•  sounds  good 

•  very  effective 

•  In  the  soft-skill  area,  we  believe  that  a  maximum  exposure  to 
^igulty  and  uncertainty  provides  a  maximum  gain  in  training. 

•  i  very  much  like  the  approach  and  rationale....  This  work  should 
have  been  started  five  years  ago I 

•  Excellent 

•  There  is  no  doubt  that  simulation,  as  defined,  has  a  prominent 
place  in  the  'soft-skill'  area. 

•  Z  like  it  very  much  and  we  have  gone  in  this  direction. 

•  I  think  It  would  be  a  100%  Improvement! 

•  Excellent!  I  favor  simulation,  but  above  all,  I  support  hands-on 
training,  l.e.,  using  manual,  regulations,  etc.  Not  memorizing  a 
' laundry  list ' .  Your  Ideas  are  great ! 

«  The  fact  that  something  Is  different  about  the  kinds  of  skills  or 
performance  factors  required  in  any  field  of  endeavor  suggests 
that  better  ways  and  means  to  identify,  structure,  and  organize 
such  data  must  be  developed. 

•  It  Is  most  welcome  to  see  an  effort,  the  results  of  which  will 
assist  In  the  systematic  design  of  courses  In  the  'soft-skills' 

area. 

•  ...an  Ideal  approach  to  soft-skill  education.... 

•  Appears  to  be  a  reasonable  approach. 

•  Generally  very-appropriate.... 

•  Tour  analysis  Is  correct. 

•  This  appears  to  be  a  realistic  approach. 

•  I  heartily  support  your  effort  In  this  area. 

■  I  think  the  above  approach  Is  realistic  and  a  definite  Improvement 
over  simply  memorizing  concepts,  principles,  etc. 

•  Interesting. . . . 
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Reservations  expressed  by  35  respondents  '^he  question:  What  do  you 
think  about  tha  above  approach  to  trdning  it  eoft-ekill  area? 

*  Where  will  we  get  sufficient  training  t  '-ae? 

*  More  experienced  instructors  will  be  required: 

a.  Authoritarians  can't  handle  change  from  teacher-  to 
student-  centered  Inscructlon — 'bull  sessions'  will 
appear  to  be  a  waste  of  time. 

b.  Extensive  training  of  instructors  is  required,  but 
where  will  we  get  .jjfflcient  training  time? 

c.  The  number  of  small  group  'facilitators'  required  is 
a  luxury  that  most  schools  cannot  afford,  especially 
with  manpower  restrictions. 

d.  Success  depends  on  quality  of  Che  instructor  or 
learning  facilitator,  thus  Instructor  selection 
becomes  a  problem. 

*  Overuse  of  one  method  is  a  possible  consequence. 

*  Army  simulation,  at  present,  is  not  emergent  (amb Iguous /uncertain) . 
Simulation  'samples'  of  real-life  are  often  applied  'verbatim'  by 
young  officers  to  real-life  problems. 

*  When  documentation.,  doctrine  and  Job  standards  are  weak— simulation 
presents  problems. 

*  ...this  approach  must  be  used  in  conjunction  with  field  observa¬ 
tion  in  order  to  be  effective. 

*  The  solving  of  many  small  problems  Is  not  always  tantamount  to 
solving  larger,  more  complex  ones. 

*  The  evaluation  procedure  remains  highly  subjective: 

a.  Can  a  simulation  be  both  sufficiently  controlled  for 
answers  so  chat  evaluation  is  valid,  yet  offer  the 
range  of  variables  even  approaching  that  of  real-life 
battlefield  problems? 

b.  Peer  evaluation  is  'blind  leading  the  blind* — should 
also  vae  superiors  or  subordinates  as  in  real-life 
situations . 

c.  Too  many  variables  in  situations  addressed,  thus 
quantitative  measures  will  probably  be  fruitless 
and/or  meaningless. 

*  Numerous  ’hard  skills'  are  embedded  within  what  we  are  now 
referring  to  as  'soft-skill*  courses.  One  must  know  both 
how  to  design  a  bridge  and  how  to  organize  and  supervise  his 
resources  to  construct  that  bridge. 

*  While  not  denying  Che  validity  of  self-paced  methods,  there  is 
a  point  at  which  caution  needs  to  be  exercised  in  prescribing 
conceptual  data  as  a  prerequisite  of  group  work.  If  conceptual 
material  is  reserved  for  a  time  before  Che  group  is  formed,  as 
seems  implied  in  the  survey,  the  group  loses  a  primary  vehicle 
for  developing  inter-personal  skills  and  for  internalizing  the 
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of  the  skllla  they  wiet  ply:.  is 

especially  true  of  concepts  that  they  hecoMj|%MstaiHsi3il3iBful  and 
useful  when  they  have  been  'dlsoovered'  In  ^yparomaaosff  'doing* 
Bonet^ing  rather  thl^l  in  reading  about  theli.  '’^tlABeuphei;^  become 
■ore  t^^en  ideas,  but  the  basis  upon  which  aotineai.>iai*  eAlten,  and,  as 
far  af,,^  possible,  should  be  dealt  with  in  avgi;in4)p/vhel«  Ideas  can 
be  pi^, jessed,  bounced  off  others,  trjjad' out,. ,pxidi<i£(inaldgm 'discovered' 

.^as  essential  to  effective  per f ormancie  «£' n-  tasi:.*  ,3gBaiA&  process, 
"Interpersonal  skills  grow.  rssiisvlsos^ls 

Secondly,  if  the  use  of  groups  as  learning  teams  becomes  the 
media  through  which  learning  happens,  es  is  implied^. the. course 
>  dliign%T-  must  be  vety  careful  abbut 
Indlvl^uhllzed/self-'paced  instruction  t6'  be  ‘  a^Compl 
ihf  iigip^up.  One  of  the  streng^ths  of  ui»iag  a  gfoup'ak-i'ibSartiing  team 
..|d 'tbi^j_  the  cumulative  cognitive  knowledge  ef.itha.totilagroup  is 
always' superior  to  the  knowledge  of  any  single  indlvld^l  ip  the 
baw.  •5';yr<group'--<and  frequently  of  the  instructiit  .as  well)  J^l’ifo^pKscribe 

::  .'<ri  leognlCive  data  to  be  learned  outside  the  groups  ^^^zobt^e  instruction 
Bi'-j  ov  of  a  significant  resource— the  group  as  instructor it  is 

true  that  at  times  the  situation  may  dictate  that^iWoivxng  several 
.^tudsQts  in  Individualized  study  is  more  efficient  u>eba»8beqnlring 
rthe  wlwle  group  to  deal  with  it  on  a  basic  level,  there  site  trade¬ 
offs  which  must  be  considered  in  making  the  choice, 

1)  As  the  group  assumes  responsibility  for  te«Ch<tng3ML  its 

BMmbers,  interpersonal  skills  are  developed  which  are  ^uch  igore 
significant  than  the  time  spent  in  bringing  the  less  %i(^edgeable 
.Student  up  to  their  level.  .<  .v.>enl  * 

2)  As  the  group  teaches  data,  no  matter  how  baslidd-^  learns 

the  data  in  greater  depth.  jc  Ic 

,  Finally,  data  learned  in  preparation  for  an  'emergap£»'  situation 
may  be  data-  which  must  be  re-leamed.  It  goes  withoius  sailing 
that  the  most  efficient  way  for  cognitive  Informatl, on  to  .be  .learned 
and  retained  is  to  Introduce  it  at  the  point  of  much  as 

is  pobslble,  the  learning  situation  should  dictate;  ^at  point 

'  cognitive  data  is  needed  and  introduced.  To  prescriVe'"^^^ as  a 
prerequisite  to  a  task  objective  may  be  to  put  it  po\nt  least 

to  the  student.  As  far  as  is  possible,  the,  praiqidSie  t>f  presclbing 
.cognitive  data  as  a  prerequisite  to  a  tralnlng/learhingtebjective 
ahculd  be  minimized.  ;i  -il98 

On  another,  but  related  topic,  the  concept  of  traimifa^  objectives 
in  Systems  Engineering  must  be  addressed.  Systems  En^in^ripg  seems 
to  heve  been  developed  to  respond  best  to  hard  sklll^.'^^^fnus,  the 
j^ff^pccentratlon  on  ’tasks'  is  appropriate.  In  the  soifatrekAdl  Area,  the 
,C9j^c^tratlon  on  'tasks'  seems  superfluous  and  superfijblbd.  Particu¬ 
larly  in  the  C-22  program  (Officer  Carrer  OouceC)  ,oiheae^clf ic  tasks 
are  vague.  Those  tasks  which  can  be  defined  and  mea8u;e^r''^®ry  precisely 
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iMually  the  least  significant  and  could  aS '  CaAilVDe. learned  in 
3?0JTi«'  lAich  more  significant  are  tasks  belonging  t&^inl  Ifiectlve 


ifoiriv  iTtdomAlAe.  and  lizpactlng  on  his  attltudlnal  ebviron&i^tt^^nise  tasks 
baw  .fidAfy  dlfinltlon  except  in  such  vague  terms  as 


V-47 


•tc.,  and  are  hardly  the  kind  of  tasks  for  which  precise  criteria 
or  atandard*  can  be  established.  It  is  v  ■  ')oinlon  that  the  concept 
of  tasks  ee  labraced  in  systecit  englne«,x  aust  be  modified  by 
Incorporat^BlIf'. attltudlnal  criteria.  Thus,  at  least  for  C-22  students, 
tha  question  which  systems  engineering  must  ask  Is  not,  'What  does 
the  student  need  to  be  able  to  do,'  but,  'What  kind  of  a  person  are 
«e  trying  to  develop? ' .  Both  curriculum  development  and  methodology 
must  converge  on  this  modification  If  we  are  to  produce  someone  to 
'manage,'  Supervise,'  etc.,  with  any  degrae  of  sensitivity  and 
ef f actlvanass . 

Answers  by  35  respondents  to  the  question:  Vhat  other  approaohee  do 
you  reoonnend  or  feel  that  are  inportmt  for  thie  purpose? 

*  Certainly  all  methods  and  media  should  be  considered  lest  we  go 

as  far  in  this  direction  as  we  were  during  the  teacher- centered  area. 

*  Lectures,  conferences,  case  studies,  seodnars,  panels,  programmed 
Inatructlon— all  have  their  place.  The  objective  Is  paramount. 

Schools  should  usa  methods  that  will,  for  them,  best  achieve  the 
objective. 

*  Use  the  following  approaches:  case  method  (real  problem  presented 
to  students  for  a  decision) ;  In-Basket  (several  mini-problems  are 
given  to  students  for  a  decision);  Role-Playing;  Debates;  Practical 
Exercises;  Mini-Teaching  (e.g.,  briefings,  and  teaching  and  critiques). 

*  Focus  on  broad  job  functions. 

*  Instead  of  'tasks'  -use  subject  areas  (i.e.,  personnel  management, 
military  justice,  training  management,  etc.)  to  obtain  a  consensus 
of  opinion  on  the  need  for  instruction  In  each  area,  then  iise  the 
experience  and  expertise  of  the  appropriate  school  staff  and  faculty 
to  determine  the  subject  area  scope. 

*  Have  students  devise  individual  solutions  to  a  series  of  realistic 
field  problems  In  the  absence  of  totally  adequate  resources  and/or 
time.  A  similar  technique  could  be  employed  with  student  groups, 
as  appropriate. 

*  Emphasise  face-to-face  discussions  with  graduates  in  Che  field  to 
determine  their  needs  and  i^at  Is  expected  of  them.  Continue  with 
self-tutor,  self-paced  and  Let's  go  Modem!  Use  casettes,  tapes, 
film  as  modem  civilian  schools  are  using. 

*  Employ  student-centered,  performance-related  training. 

*  Develop  some  situational  performance  tests,  oriented  to  what  the 
officer  would  do  on  the  job  and  preaent  them  to  the  student  in  as 
elosa  to  the  'job  environment'  as  you  can  make  it. 

*  In  many  Inatances,  dua  to  limited  resources,  It  Is  possible  for 
Che  student  to  be  put  In  'planning'  situations  raqulrlng  studmts 
to  prepara  a  plan  for  exacutlon  of  some  defined  objective  In  which 
limitations  are  imposed  on  the  range  of  his  decision-making  and 
limited  resources  made  available  for  allocation. 
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For  soft-ckill  couroos  such  as  leadership  courses,  it  is  very 
laportant  to  design  the  task  inventories  carefully.  They  need 
to  be  aequentlally  arranged  from  the  visualised  start  of  an 
operation  or  job  requlreacnt  for  an  expected  job  situation  through 
to  completion  of  the  job  requirement.  Systematic  breakdown  of 
tasks  are  Important.  It's  better  to  be  In  too  much  detail  than 
not  enough.  Later  aummarlsatlons  can  be  made  as  necessary. 

Another  possibility  (which  should  not  be  used  to  excess)  Is 
learning  by  exception;  l.e.,  attempting  to  perform  the  function 
without  any  previous  Instruction  on  basic  principles  or  procedures. 
This  Is  followed  by  a  critique  which  brings  out  suggestions  for 
Improvement  and/or  points  out  actions  cosndtted  or  omitted  which 
were  detrimental  to  procedure,  or  technique  which  surpasses  the 
'school  solution*.  If  not,  the  'school  solution*  can  always  be 
contrasted  with  his,  showing  the  rational  which  supports  the 
'school  solution'. 

Ike  Group  Process  Flan  (Indiana  Plan)  of  the  USA  Chaplain  School. 
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PACT  SHEET 

THE  REDESICaiED  CHE.'OiCAL  QFFICLR  ADVANCED  COURSE 

Beginning  vlth  the  class  opening  In  Sapteaber  1971,  the  Chemical  Officer 
Advanced  Course  has  undergone  significant  change.  The  course  is  now  de¬ 
signed  to  encourage  maximum  student  I'l-rticipation,  response,  and  growth 
by  helping  him  to  learn  to  cope  with  realistic  and  relevant  situations 
in  an  atmosphere  that  allows  optimum  latitude  for  the  exercise  of 
initiative.  To  prepare  the  graduate  for  continuing  growth,  much  greater 
emphasis  has  been  placed  on  the  fundamentals:  the  communicative;  human 
relations;  quant Itatlve/computatlonal;  and  conceptual /problem  solving 
skills. 

nie  curriculum  Includes  the  essential  professional  and  technical  knowledge, 
that  is;  principles,  doctrine,  concepts,  techniques,  and  procedures. 

What  has  changed  is  the  orientation  of  the  course,  the  curriculum,  and 
the  faculty.  No  longer  is  the  instruction  theory  -  and  instructor- 
centered.  It  is  now  learning-  and  student-centered.  The  philosophy 
that  fostered  the  selection  and  teaching  of  material  it  was  assumed  the 
student  needed  to  know,  followed  by  practical  exercises  that  directed 
the  student  toward  narrow,  almost  specified,  solutions,  with  little  or 
no  margin  for  error,  has  been  done  away  with.  It  has  been  replaced  with 
a  philosophy  that  requires  the  faculty  to  respond  to  the  student’s  real 
needs  by  finding,  or  designing,  realistic,  relevant  situations  that  the 
student  can  imagine  himself  facing  in  the  real  world.  The  faculty  gives 
the  student  only  the  minimal  instruction  that  is  pertinent  to  the 
situation.  Great  dependence  is  placed  on  the  student  identifying  the 
problems  inherent  in  the  situations  through  his  own  participation  and 
activity,  learning  by  doing,  individual  effort,  peer  instruction,  and 
self  pacing.  What  might  be  called  the  "whole-school  concept"  is  employed. 
Bie  student  calls  upon  the  faculty,  classmates,  library,  staff  and  support 
activities  (to  include  the  Post  staff  and  facilities),  and  whatever  out¬ 
side  sources  he  may  need.  He  is  given  time  and  reason  to  use  them.  The 
lecture  and  conference  are  rare  (except  for  certain  specific  technical 
subjects  such  as  Prefix  3);  the  seminar  is  more  common,  but  is  kept  to 
a  minimum.  Attendance  at  these  is  voluntary. 

!Rje  vehicles  used  are  the  various  command  levels  of  the  Army:  company/ 
battalion,  brigade,  division,  installation  (CONUS  Base)  and  hl^er  head¬ 
quarters  (to  include  high  level  logistics).  These  levels  (at  each  of 
which  the  class  spends  5“6  consecutive  weeks)  allow  a  design  of  the 
learning  process  which  progresses  from  attacking  relatively  siiiple 
specified  problems  to  more  highly  complex  situations  requiring  identi¬ 
fication  of  problems,  followed  by  analysis  and  .development  of  workable 
solutions.  It  should  be  understood  that  these  levels  are  merely  vehicles 
in  the  learning  process  and  are  not  designed  to  emphasize  the  organiza¬ 
tions  selected.  Ihey  pi-ovide  a  device  for  integrating  learning  in  a 


logical  progression,  vhlle  at  the  same  time  Illustrating  problems  and 
procedures  similar  to  those  that  may  be  encountered  In  later  assignments. 
Realism  and  relevance  depend  on  the  careful  selection  of  problems  and 
situations.  As  far  as  practicable  these  have  been  extracted  from 
current  records,  documents,  and  files.  Some  of  the  actual  reply  or 
resjKinse  documents  are  available.  School  solutions  have  for  the  most 
part  been  eliminated  In  that  the  overriding  objective  Is  not  a  solution, 
per  se,  but  the  development  of  the  student's  ability  to  cope  with  problems 
anS  arrive  at  logical,  sound  approaches  to  solutions,  recoonaendatlons, 
or  decisions. 

The  class  Is  organized  Into  small  work  groups,  membership  of  vhlcb  Is 
changed  at  each  level.  The  class  meets  as  a  vhole  only  occasionally 
for  camnon  instruction.  A  faculty  consultant  (normally  an  experienced 
major)  Is  available  to  each  group  at  eiU  times  to  answer  questions, 
provl^  guidance,  closely  observe  Individual  and  group  action  and  Inter¬ 
action,  and  provide  evaluations  of  the  students. 

Student  evaluation  is,  for  the  most  part,  subjective,  based  on  per¬ 
formance.  Ihe  student  is  rated  on  what  he  can  do,  and  how  well  be  does 
It,  not  on  what  he  knows  as  deteimined  by  objective  recall  or  recognition 
tests.  Performance  inthe  course  Is  based  on  written  and  oral  presenta¬ 
tion  of  solutions  and  the  work,  logic,  and  breadth  and  depth  of  overall 
understanding  that  has  gone  Into  these  solutions.  The  system  Is 
essentially  macro-evaluation  as  opposed  to  the  micro-evaluation  common 
to  objective-type  evaluations.  Students  are  scored  on  a  pass-fall  basis 
for  each  performance  with  provision  for  recognition  of  distinguished 
performance. 


ORADUAOE  PROFILE 


THE  CHEMICAL  OFFICER  ADVANCED  COURSE  GRADUATE  MUST  EE: 

1.  PROFICIBRT  IN  TECHNICAL  ASPECTS  AND  APPLICATIONS  OF  CBR. 

2.  ABLE  TO  IDEHTIFr  AND  ANALYZE  COMPLEX  PROBLEMS. 

3.  ABLE  TO  THUnC  IMAGINATIVELY  AND  IN  THE  BROADEST  RANGE  POSSIBLE. 

4.  CONSCIOUS  OF  THE  ISSUES  AND  PROBLEtG  OF  THE  ARMY  AND  SOCIETY. 

3.  SKILLED  IN  INTERPERSOKAL  RELATICXIS. 

6.  RELAXED  IN  THE  PRESENCE  OF  HIGH-RANKING  OFFICERS  AND  OFFICIALS. 

7.  ABIE  TO  DEMOHSTRAOE  HIGH  PERSUASIVE  ABILITIES. 

8.  ABLE  TO  C0M4UNICA1E  EFFECTIVELY—  ABLE  TO  VRIIE  AND  SPEAK  IN 
SUCH  A  WAY  AS  TO  PRESET  COMPLEX  ISSUES  CLEARLY. 

9.  FAMILIAR  WITH  PROBLEMS  AND  PROCEDURES  AT  VARIOUS  CCMfAND  LEVELS. 
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D  .  M»ier  is  currently  assigned  as  Project  Director  for  Research  on 
Enlisted  Training  and  Glassification,  U.  S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences. 

He  received  the  PhD  in  psychology  from  Purdue  University  in  1959.  After 
a  period  with  Army  personnel  research,  he  joined  Educational  Testing 
Service,  Princeton,  New  Jersey,  and  remained  until  1965.  While  with 
Educational  Testing  Service  he  conducted  research  on  the  development  and 
evaluation  of  self- instructional  programs.  His  work  there  resulted  in 
several  reports  and  a  book  entitled,  A  Guide  to  Evaluating  Self-Instructional 
Programs. 

Since  joining  the  Army  Research  Institute,  he  has  been  primarily  responsi¬ 
ble  for  developing  the  new  Army  Classification  Battery  and  aptitude  area 
system,  which  is  scheduled  for  operational  use  early  in  1973.  He  has  con¬ 
ducted  research  on  the  effects  of  educational  level  on  the  validity  of  the 
Army  Classification  Battery  and  on  the  effectiveness  of  Army  classification 
tests  for  blacks  and  whites.  In  the  future.  Or.  Maier  will  asstime  greater 
responsibility  for  individual  MOS  training  in  addition  to  his  work  on  classifi¬ 
cation  testing. 


AN  IMPROVED  ARMY  CLASSIFICATION  BATTERY 

Milton  H.  Maier 
U.  S.  Army  Research  Institute 

Each  of  the  armed  services  is  faced  with  the  problem  of  selecting, 
classifying,  and  assigning  to  training  and  jobs  large  number  of  young 
men  who  enter  the  service.  Most  of  the  men  have  limited  work 
experience  and  little  technical  training;  thus  little  information  is  readily 
available  about  the  jobs  they  cam  best  fill  while  on  active  duty.  Selec¬ 
tion  and  classification  tests  have  been  developed  and  used  over  the  years 
to  measure  the  potential  of  the  young  men  to  perform  in  the  large  variety 
of  jobs  open  to  recruits.  In  the  Army  the  nixmber  is  several  hundred 
jobs.  The  tests  are  an  efficient  and  accurate  means  of  assessing  the 
potential  to  succeed  in  each  of  the  jobs  or  associated  training  courses, 
and  permit  an  effective  match  between  the  needs  of  the  service  and  the 
capabilities  of  the  new  recrxiits. 

The  Army  Classification  Battery  (ACB)  has  been  used  for  over  20 
years  in  assigning  men  to  their  job  training  courses.  As  the  Army 
moves  from  an  induction  input  to  an  all  volunteer  input,  the  ACB  will 
be  used  increasingly  to  help  make  selection  decisions  as  well.  In  the 
various  enlistment  options  available,  mental  standards  are  set  in 
terms  of  the  Armed  Forces  Qualification  Test  (AFQT)  and  aptitude  area 
scores,  derived  from  the  ACB  or  its  counterpart,  the  Army  Qualifica¬ 
tion  Battery  (AQB).  Using  the  test  battery  for  selection  imposes  some¬ 
what  different  requirements  than  using  it  only  for  classification.  If 
the  tests  were  used  only  for  classification  and  assignment,  then  the 
differences  between  the  aptitude  scores  would  be  the  critical  factor.  We 
would  want  to  know  if  the  man  had  more  potential  as,  say,  a  mechanic 
or  clerk;  but  because  he  already  has  been  selected  into  the  service  on 
the  basis  of  another  test,  such  as  the  AFQT,  we  would  assume  that  he 
is  at  least  minimally  qualified  in  both  areas.  If  the  test  battery  is  used 
for  both  selection  and  classification,  then  we  want  to  know  not  only  the 
differences  in  potential,  but  also  whether  he  is  mentally  qualified  in  the 
different  areas. 


The  distinction  between  selection  and  classification  was  brought  home 
forcibly  to  the  Army  when  the  mental  standards  were  lowered  for  Project 
100,  000  in  1966.  Up  until  that  time  all  the  input  had  the  minimal  levels 
of  literacy  and  general  mental  ability  because  the  mental  standards  were 
high.  The  aptitude  area  scores  obtained  from  the  ACB  were  used  to 


reveal  the  job  areas  in  which  the  it  n  could  be  .>ost  utilized.  Men 
could  be  assigned  to  the  differenct^  areas  on  the  ba-^is  of  the’r  apti¬ 
tude  area  scores  with  reasonable  confidence  that  they  would  perform 
satisfactorily. 

When  selection  standards  were  lowered  to  accept  men  with  lower 
mental  ability,  the  level  of  general  ability  was  changed  sufficiently 
to  create  probler  Some  Army  schools  were  receiving  too  many 
men  who  could  not  absorb  the  highly  technical  material  because  they 
were  too  low  on  general  ability.  We  developed  an  interim  solution  to 
make  the  aptitude  area  scores  more  suitable  with  the  new  levels  of 
input,  but  it  was  not  implemented  operationally  because  of  the  manpower 
implications.  We  were  developing  a  new  classification  battery  at  the 
time,  which  is  expected  to  be  ready  for  operational  use  early  in  1973. 
The  aptitude  area  scores  obtained  from  the  new  ACB  are  more  suitable 
for  the  dual  purpose  of  selection  and  classification. 

One  need  for  a  new  system  to  select  and  classify  Army  enlisted  men 
became  apparent  with  the  lowering  of  mental  standards.  Another  need 
for  a  modification  of  the  test  battery  arose  from  the  passage  of  time  and 
the  associated  changes  in  our  culture  and  the  Army  training  courses. 
Some  of  the  test  items  date  back  to  the  1950*6,  and  some  of  these  seem 
out  of  place  today.  For  example,  the  German  general,  Rommel,  The 
Desert  Fox,  no  longer  receives  much  popular  attention,  and  television 
picture  tubes  have  changed  design  as  compared  to  20  years  ago.  In 
addition,  the  Army  technology  has  changed  with  more  sophisticated 
equipment.  The  training  requirements  have  been  increased  to  enable  the 
men  to  cope  with  the  more  technical  machines  and  concepts. 

Starting  over  a  decade  ago,  we  began  an  extensive  research  pro¬ 
gram  to  develop  a  new  version  of  the  ACB.  The  research  objectives 
were  to  update  the  tests  that  were  becoming  obsolescent,  to  find  con¬ 
tent  more  suitable  to  the  newer  types  of  job  training  courses  and  to 
find  new  kinds  of  tests  that  would  increase  the  predictive  accuracy  of 
the  battery. 

The  research  prograun  was  successful  in  that  a  new  version  was 
developed  which  does  meet  more  effectively  the  needs  of  the  modern 
Army.  The  relationship  between  the  tests  in  the  old  and  new  ACB  is 
shown  in  Table  1. 
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COMBAT  SCALE  CCC)  CLASSIFICATION  INVENTORY  CCI)  UPDATED  AM)  SHORTENED 

ATTENTIVENESS  SCALE  CCA)  ADDED 

ELECTRONICS  SCALE  CCE)  ADDED 

MAINTENANCE  SCALE  (CM) 


The  New  Army  Classification  Battery 

General  Ability  Tests.  The  new  ACB  has  fjve  tests  of  general 
abi'ity,  three  common  to  the  new  and  previous  battery,  and  two 
added  tests,  Mathematics  Knowledge  and  Science  Knowledge.  The 
Word  Knowledge  and  Arithmetic  Pvcasoning  tests  are  changed  from  the 
original  only  in  having  been  shortened  to  provide  more  efficient  mea¬ 
surement.  The  General  Information  test,  updated  and  shortened,  has 
shifted  from  its  function  as  primarily  a  combat  selector  to  serve  as  a 
measure  of  the  general  ability  required  of  good  performers  in  selected 
noncombat  MOS  as  well  as  in  artillery.  The  Mathematics  Knowledge 
and  Science  Knowledge  tests  were  added  to  expand  coverage  in  this 
important  aptitude  domain.  Each  of  the  five  tests  measures  a  different 
aspect  of  general  ability.  The  Word  Knowledge,  Arithmetic  Reasoning, 
and  General  Information  tests  cover  skills  and  knowledge  that  can  be 
acquired  in  or  out  of  school.  The  other  two  tests  cover  abilities  taught 
in  formal  school  courses.  All  five  tests  measure  aptitudes  required 
in  a  wide  variety  of  jobs  and  siutations. 

Mechanical  Ability.  Four  mechanical  ability  tests  are  included  in 
both  batteries.  The  Automotive  Information  Test  was  shortened  for  the 
new  battery.  The  Shop  Mechanics  Test  was  dropped  and  replaced  by 
Trade  Information.  Content  of  the  Electronics  Information  Test  was 
updated.  The  Mechanical  Aptitude  Test  was  updated  and  the  title  changed 
to  Mechanical  Comprehension,  The  new  tests  have  the  advantages  that 
the  content  is  up  to  date,  the  tests  are  more  valid,  and  all  are  shorter. 

Perceptual  ability.  The  three  tests  of  perceptual  ability  require 
no  reading  or  writing  skills  but  do  require  ability  to  perceive  certain 
kinds  of  stimuli- -geometrical  patterns,  and  auditory  and  visual  symbols. 
The  new  version  of  the  Pattern  Analysis  Test,  which  requires  visualiza¬ 
tion  of  three-dimensional  form,  is  shorter  than  the  previous  one.  The 
Army  Radio  Code  Aptitude  Test  has  a  new  title,  Auditory  Perception, 
but  otherwise  remains  the  same.  The  more  inclusive  title  reflects  the 
finding  that  the  test  is  useful  for  jobs  other  than  radio  operator --jobs 
that  require  the  ability  to  listen  attentively.  The  Army  Clerical  Speed 
test  was  replaced  by  Attention  to  Detail,  which  is  more  widely  useful 
and  easier  to  administer. 

Self-Description  Test.  An  expanded  version  of  the  Classification 
Inventory,  long  used  to  identify  men  who  will  make  good  combat  soldiers, 
was  introduced.  Four  separate  measures  arc  obtained  from  this  test: 
Scale  CC  corresponds  to  the  previous  Classification  Inventory  score  used 
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to  identify  combat  infantrymen,  but  it  has  been  updated  and  shortened. 
Scale  CA  is  a  measure  of  attentiveness,  a  useful  predictor  for  a 
variety  of  jobe- -clerical,  artillery,  missile  crewman,  for  example. 

Scale  CE  (electronics)  and  Scale  CM  (maintenance)  are  related  to 
specific  job  families;  both  help  identify  repairmen  who  will  be  successful 
in  the  relevant  area. 

Grouping  the  MOS 

The  development  of  new  tests  does  not  by  itself  result  in  a  new 
classification  system.  A  critical  component  is  that  of  grouping  the  jobs 
into  relatively  homogeneous  clusters  or  families.  As  already  mentioned, 
there  are  several  hundred  jobs  potentially  open  to  the  Army  recruit,  and 
some  of  these  jobs  are  more  alike  than  others.  The  Army  philosophy, 
with  some  support  from  research  data,  is  that  the  jobs  should  be  grouped 
into  a  mainageable  number  of  categories,  which  is  in  the  range  of  about 
8  to  12.  The  jobs  within  a  group  should  require  similar  skills  and 
aptitudes  for  success,  and  be  as  different  as  possible  from  the  jobs  in 
other  families.  Thus  the  electronics  repair  jobs  form  one  cluster,  which 
is  different  from  the  clerical -administrative  jobs. 

The  grouping  of  the  jobs  was  accomplished  by  computing  the  validity 
of  each  test  for  each  sample  separately.  Since  we  had  over  30  tests 
and  100  samples,  there  were  well  over  3000  validity  coefficients.  We 
examined  the  validity  profiles  for  each  sample,  and  grouped  those  jobs 
together  that  tended  to  require  the  same  aptitudes  and  interests  for 
success.  We  had  some  help  in  this  grouping  process  from  the  structure 
of  Army  jobs  already  in  existence.  We  found  that  with  some  excep¬ 
tions  we  could  base  our  grouping  on  that  used  operationally. 

The  final  result  of  grouping  the  jobs  is  shown  in  Table  2.  We 
ended  up  with  nine  MOS  groups,  shown  in  the  left  column;  some  repre¬ 
sentative  jobs  in  each  group  are  shown  in  the  right  column.  Each  of 
these  groups  was  developed  on  an  empirical  basis.  The  MOS  were 
grouped  together  only  if  our  data  showed  that  they  were  similar  in 
terms  of  the  interests  and  aptitudes  required  for  success  and  different 
from  the  MOS  in  other  groups. 

The  first  MOS  group,  called  CO  for  Combat,  includes  the  infantryman, 
armor  crewman,  and  combat  engineer.  The  second  group,  FA  for  Field 
Artillery,  includes  the  field  cannon  and  rocket  artillery  jobs.  The  third 


TABLE  2 


REPRESENTATIVE  JOBS  IN  NEW  MOS  GROUPS 


MOS 

GROUP 

REPRESENTATIVE  JOBS 

COMBAT 

(CO) 

INFANTRY^  ARMOR^  COMBAT  ENGINEER 

FIELD  ARTILLERY 

(FA) 

FIELD  CANNON  AND  ROCKET  ARTILLERY 

ELECTRONIC  REPAIR 

(EL) 

MISSILES  AND  AIR  DEFENSE  REPAIRMEN, 

TACTICAL  ELECTRONIC  AND  FIXED  PLANT 
COMMUNICATIONS  REPAIRMEN 

OPERATORS/POOO 

(OF) 

MISSILES  AND  AIR  DEFENSE  CREWMEN, 

DRIVER,  FOOD  SERVICES  ^ 

SURVEI LLANCE/COMMLNICATIONS 

(SC) 

TARGET  ACQUISITION  AND  COMBAT  SURVEILLANCc, 
AND  COMMUNICATION  OPERATIONS 

MECHANICAL  FAINTE»MNCE 

OH) 

MOTOR  AND  AIRCRAFT  MAINTENANCE,  RAILWAYS 

GENERAL  MAINTENANCE 

(GM) 

CONSTRUCTION,  UTILITIES,  CHEMICAL, 

MARINE,  PETROLEUM 

CLERICAL 

(CL) 

ADMINISTRATIVE,  FINANCE,  SUPPLY 

SKILLED  TECHNICAL 

(ST) 

MEDICAL,  MILITARY  POLICE,  DATA  PROCESSING, 
AIR  CONTROL,  TOPOGRAPHY  AND  PRINTING, 
INFORmTiON  AND  AUDIO  VISUAL 

group,  EL  for  Electronica  Reparis,  includes  all  electronics  and 
electrical  maintenance  MOS  ^  nparate,  but  out  data  did  not  support 
such  a  distinction,  and  we  combined  them. 

The  fourth,  group,  OF  for  Operators -Food,  includes  a  seemingly 
diverse  collection  of  jobs:  missile  crewmen,  cooks,  and  drivers.  The 
grouping  emerged  from  our  data,  and  their  common  feature  seems  to 
be  a  requirement  for  a  concern  with  details  (reflected  by  the  Attentive¬ 
ness  measure)  and  ability  to  handle  objects  easily  (reflected  by  the 
Automotive  Information  test. )  The  next  group,  SC,  for  surveillance 
and  communications,  includes  radio  operators,  communication  center 
specialists,  and  switchboard  operators.  The  MOS  involve  receiving 
and  processing  information;  the  common  element  seems  to  be  a  require¬ 
ment  for  perceptual  ability,  both  auditory  and  spatial. 

There  are  two  maintenance  groups,  MM  for  Mechanical  Maintenance 
and  GM  for  general  maintenance.  The  MM  group  includes  motor 
mechcinics,  aircraft  maintenance  and  railway  jobs.  GM  covers  a  variety 
of  jobs,  such  as  construction,  utilities,  marine,  chemical,  and 
petroleum. 

The  final  two  groups,  CL  for  clerical  and  ST  for  skilled  technical, 
are  familiar  to  Army  personnel.  CL  includes  the  administrative,  finance, 
and  supply  jobs.  ST  is  similar  to  the  old  GT  area,  and  includes  medics, 
military  policemen,  and  intelligence  specialists. 

The  New  Aptitude  Area  Compositer 

The  final  step  in  developing  a  new  classification  system  is  to  find 
the  weights  to  assign  to  each  test  for  each  MOS  group.  The  weights 
were  obtained  by  determining  which  tests  contributed  most  to  pre¬ 
dicting  success  in  each  area.  In  selecting  the  tests,  we  first  selected 
the  test  that  was  most  valid;  then  we  added  the  test  that  made  the  second 
largest  contribution  to  validity.  We  continued  the  selection  process 
until  the  remaining  tests  made  little  contribution  to  increasing  the 
accuracy  of  prediction.  Generally  we  had  to  select  4  or  5  tests  for 
each  MOS  before  we  exhausted  the  validity  of  the  battery;  in  two  of  the 
areas  we  needed  only  three  tests.  The  tests  selected  for  each  MOS 
area  were  assigned  a  weight  of  one;  we  found  through  extensive  simula¬ 
tion  studies  on  the  computer  that  simple  unit  weights  were  as  effective 
as  more  elaborate  weighting  schemes.  Those  tests  not  selected  were 
assigned  a  weight  of  zero.  The  tests  used  for  the  MOS  groups  are 
shown  in  Table  3. 
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Table  3 

.  Mur  APTITODB  ARBB  COMPOSITES 


Test  Aptitude  Area  Copipositcs 


^Ceoeral  ‘Ability  Testa 

.  CO 

FA 

EL 

OF 

sc 

MH 

GM 

CL 

ST 

GT 

Arithmetic  Reasoning 

•CAR) 

AR 

AR 

/ 

AR 

/ 

AR 

AR 

AR 

AR 

AR 

General  Information 

(Cl) 

ci 

GI 

Mathematics  Knowledge 

(MK) 

• 

MK 

, 

MK 

MK 

Word  Knowledge 

(WK) 

HK 

WK 

VK 

Science  Knowledge 

<SK) 

, 

8K 

SX 

ifechanical  Ability  Tests  . 

• 

Trade  Information 

ai) 

TI 

TI 

TI 

Electronics  Information 

(El) 

El 

El 

BI 

, 

Mechanical  Comprehension  ^IC) 

MC 

MC 

MC 

Automotive  Information 

(AI) 

AI 

AI 

AI 

Perceptual  Ability 

i  .  • 

1 

1 

! 

• 

Pattern  Analysis 

(PA) 

PA 

PA 

'•  Attention  to  Detail 

(AD) 

AD 

AD 

Auditory  Perception 

(AP) 

AP 

Self  Description 

1 

, 

Combat  Scale 

(CC) 

CC 

! 

Attentiveness  Scale 

(CA) 

CA 

CA 

Electronics  Scale 

(CE) 

CE 

Maintenance  Scale 

(CM) 

j 

CM 

1 
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Our  research  showed  that  the  good  combat  soldier  needs  general  ability, 
measured  by  the  Arithmetic  Reasoning  Test,  mechanical  ability,  measured 
by  the  Trade  Information  Test,  to  handle  his  weapons  and  equipment;  per¬ 
ceptual  ability  measured  by  the  Pattern  Analysis  and  Attention  to  Detail 
tests,  to  orient  himself  in  the  terrc:?n  and  observe  his  environment,  and 
finally,  an  interest  in  outdoor  mascui^iAe  activities,  coupled  with  self- 
confidence,  measured  by  the  Combat  scale  (CC)  of  the  Classification 
Inventory. 

The  artilleryman,  in  comparison  was  found  to  require  more  mathema¬ 
tical  ability.  Therefore,  scores  from  both  the  Arithmetic  Reasoning  Test 
ajid  the  Mathematics  Knowledge  Test  enter  into  the  Field  Artillery  (FA) 
Aptitude  Area.  A  further  measure  of  general  ability  is  contributed  by  the 
General  Information  Test.  Mechanical  ability,  measured  by  the  Elect¬ 
ronics  Information  test,  and  an  interest  in  details,  measured  by  the 
Attentiveness  (CA)  scale  of  the  Classification  Inventory,  complete  the 
picture  for  the  artilleryman. 

A  similar  analysis  can  be  made  for  each  MOS  Group.  In  all  cases  the 
tests  that  were  selected  for  the  composites  made  sense  based  on  what  we 
know  about  the  jobs  in  each  group. 

A  word  should  be  said  about  the  final  composite  in  the  list,  the 
familiar  General  Technical  (GT)  Aptitude  Area,  composed  of  the  Arith¬ 
metic  Reasoning  and  Word  Knowledge  (Verbal)  tests.  In  the  old  system, 
the  GT  score  is  used  both  to  select  men  for  general  technical  MOS  and 
to  determine  which  men  are  eligible  to  take  additional  tests  such  as  the 
Officer  Candidate  Test.  The  function  of  selector  for  MOS  group  is 
shifted  to  the  ST  composite.  The  function  of  determining  eligibility  for 
additional  testing  continues  to  be  filled  by  the  combination  of  Arithmetic 
Reasoning  and  Word  Knowledge  Tests.  The  label  GT  is  retained. 

Evaluation  of  the  New  Classification  System 

The  new  Army  classification  system  was  carefully  evaluated  to 
estimate  how  much  improvement  would  be  realized  over  the  old  system. 

The  conclusions  we  have  reached  are  that  academic  attrition  in  job 
training  courses  would  be  reduced  by  about  20  percent;  that  the  number 
of  marginal  performers,  that  is,  men  who  barely  pass  the  training 
course,  would  be  reduced  by  20  percent,  and  that  the  number  of  superior 
performers  would  increase  by  15  percent.  The  procedures  were  to  run 
extensive  simulation  studies  on  our  computer;  the  details  are  given 
in  BESRL>  Technical  Research  Note  239>  In  obtaining  these  estimates 
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of  improvement  over  the  old  classification  system,  the  quality  of  men 
coming  into  the  Army  was  assumed  to  be  exactly  the  same  for  both 
classification  systems.  The  improvement  in  performance  can  be 
realir  >d  because  the  new  system  does  a  better  job  of  getting  the  right 
man  into  that  area  where  he  can  perform  best. 

The  new  ACB  would  result  in  even  greater  benefit  when  the  improved 
selection  is  considered.  Applicants  for  specific  jobs  need  to  qualify  on 
several  aptitude  scores.  Since  the  new  aptitude  scores  are  more  accurate 
measures  of  potential  than  the  old  scores,  there  is  greater  assurance 
than  men  who  meet  these  requirements  can  be  trained  to  the  point  of 
acceptable  competence  in  an  Army  job. 

In  summary,  a  new  Army  Classification  Battery  has  been  developed  that 
will  result  in  improved  selection  and  classification  of  enlisted  men,  and  it 
will  be  ready  for  operational  use  early  in  1973.  New  measures  of  interest 
and  general  mental  ability  have  been  added  to  the  battery,  and  new  com¬ 
binations  of  tests  have  been  developed  to  measure  more  accurately  the 
potential  to  perform  in  training  and  on  the  job.  The  effect  of  the  new 
system  is  to  screen  out  more  of  the  men  who  would  be  likely  failures 
and  to  utilize  more  effectively  the  talents  and  interests  of  the  men  who  are 
accepted  for  Army  service. 


DR.  C.  O.  GRAY 


Dr.  C.  O.  Gray,  who  is  currently  assigned  as  Educational  Advisor. 

U.  S.  Army  Engineer  School,  is  completing  his  30th  year  of  service  in 
the  Army  Service  School  System.  This  period  of  service  has  included 
enlisted  and  officer  instructor  and  staff  duty,  civilian  educational  staff 
specialist  assignments,  and  an  extended  period  as  Educational  Advisor 
at  the  U.  S.  Army  Ordnance  School  prior  to  ass\iming  his  current  assign¬ 
ment  at  the  U.  S.  Army  Engineer  School. 

Prior  to  this  lengthy  Army  Service  School  tour.  Dr.  Gray  completed  five 
years  as  a  public  school  teacher  in  Pennsylvania  and  New  York. 

Dr.  Gray's  academic  preparation  includes  degrees  from  New  York 
University  and  the  University  of  Maryland.  His  professional  affiliation 
includes  membership  in  Phi  Delta  Kappa,  a  national  honorary  educational 
fraternity,  and  the  American  Society  for  Engineering  Education. 
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FIELD  VISITATION  PRCXSRAM  BRIE FING 
DR.  C.  O.  GRAY 

Educational  Advisor,  U.  S.  Army  Engineer  School 


The  subject  of  this  briefing  is  the  U.  5.  Army  Engineer  School  Field 
Visitation  Program  (Slide  1).  This  program  is  defined  (Slide  2)  as  a 
programed  schedtile  of  visits  by  a  small  team  of  specialists  to  engineer 
units  worldwide  for  the  purpose  of  collecting  data  necessary  to  give  direc¬ 
tion  to  mission  efforts  and  to  assess  the  quality  of  the  school's  mission 
products  and  services.  (Slide  off).  The  specific  objectives  of  this  briefing 
are:  (Slide  3) 

-  To  enumerate  the  feedback  needs  in  an  Army  service  school 
environment 

-  To  describe  the  current  feedback  posture  in  this  environment 

-  To  identify  the  totality  of  the  USAES  field  feedback  survey  system 

-  To  examine  the  basic  elements  of  this  program.  (Slide  off). 

Field  feedback  in  an  Army  service  school  environment  is  fundamental 
(Slide  4)  so  as  to  provide  direction  for  mission  output  and  to  provide  a 
means  for  assessing  the  quality  of  mission  output.  It  will  be  noted  that 
key  words  to  be  associated  with  field  feedback  are  direction,  mis sion.  and 
assessing  the  quality.  (Slide  off).  To  give  these  terms  more  meaning  and 
to  demonstrate  their  impact  in  a  viable  feedback  effort,  I  would  like  to 
review  the  primary  missions  of  an  Army  service  school,  to  identify  feed¬ 
back  necessary  to  give  direction  to  these  missions,  and  to  address  how 
such  feedback  can  be  used  in  assessing  quality. 

The  six  Army  service  school  mission  statements  dictated  by  CON  Reg  10-4 
can  be  siunmarized  in  abbreviated  form  as  follows.  (Slide  5) 

1.  Conduct  resident  and  nonresident  training 

2.  Support  ROTC  programs 

3.  Support  unit  training,  including  expanded  reserve  component 
support. 
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4.  Assist  in  doctrine  development 

5.  Develop  training  literature  and  training  aids 

6.  Disseminate  information 

Using  this  mission  framework,  I  would  like  to  identify  with  each,  the 
contribution  of  field  feedback  to  direction  and  quality  assessment. 

The  first  mission,  (Slide  6)  conduct  resident  and  nonresident  training, 
receives  through  field  feedback  direction  related  to  -- 

-  The  verification  and  modification  of  task  inventories. 

-  The  receipt  of  input  for  the  development  of  job  standards. 

This  direction  results  in  the  development  and  conduct  of  resident  and 
nonresident  programs  that  produce  trained  students  in  job- related  skills 
and  knowledges.  The  use  of  these  skills  and  knowledges  by  the  trained 
product  on  the  job  is  assessed  through  field  feedback  by  - 

-  The  individual's 

-  The  supervisor's  and 

-  The  observer's  assessment  of  adequacy  and  competency. 

The  second  mission,  (Slide  7)  support  ROTC  programs,  receives  direction 
through  field  feedback  in  basically  the  same  manner  as  the  first  mission. 

-  The  verification  and  modification  of  task  inventories 

-  The  receipt  of  input  for  the  development  of  job  standards. 

Quality  assessment  of  the  ROTC  program  product  also  relies  on  field 
feedback  through  its 

-  Individual 

-  Supervisory  and 

-  Observer  assessment  of  adequacy  and  competency. 


The  support  of  unit  training  (Slide  8),  the'  third  mission,  gets  direction 
in  its  developmental  efforts  through  the  following  field  feedback  output 

•  The  verification  and  modification  of  unit  missions  and 

-  The  verification  and  modification  of  \init  operations. 

Assessment  of  the  quality  of  support  through  feedback  is  highlighted  by 

-  The  unit  commander's 
Key  unit  personnel  and 

-  Higher  command  assessment. 

The  fourth  mission  (Slide  9),  assist  in  doctrine  development,  relies  upon 
the  direction  received  through  fibld  feedback  in  a  manner  equal  to  if  not 
greater  than  the  previous  missions.  The  expected  feedback  direction 
includes  -- 

-  Unit  operations  assessment 

-  Operational  techniques  validity 

-  Task  inventories  verification  and  modification  and 

-  Job  standards  input. 

Quality  assessment  of  doctrine  products  uses  these  outputs  from  field 
feedback 

-  The  unit  conunander's  and 

-  The  supported  elements  assessment  of  adequacy  and  competency. 

Proceeding  to  the  fifth  mission,  develop  training  literature  and  training 
aids  (Slide  10),  we  can  identify  the  following  direction  as  emanating  from 
field  feedback  -- 

-  The  identification  of  literature  and  aids  voids 

-  The  users'  problems 

-  The  users'  characteristics  and 

-  The  users'  capabilities. 
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The  products  of  field  feedback  which  contribute  to  quality  assessment 
associated  with  this  mission  are 

-  The  individual 

-  Supervisory  and 

-  Observer  assessment  of  adequacy. 

The  dissemination  of  information  (Slide  11),  the  sixth  mission,  finds  the 
school  in  its  role  as  "Home  of  the  Engineers.  "  The  feedback  direction 
needed  to  fulfill  this  mission  include  -- 

-  A  forecast  of  projected  demands 

-  The  identification  of  unit  operational  problems  and 

-  The  identification  of  most  beneficial  information  modes. 

The  quality  of  information  furnished  can  be  assessed  through  field 
feedback  by  -- 

-  The  individual 

-  Supervisory  and 

-  Observer  assessment  of  adequacy.  (Slide  off) 

We  believe  as  a  result  of  reviewing  the  expected  direction  and  quality 
assessment  output  from  field  feedback  that  it  can  be  concluded  that  an 
Army  service  school  requires  the  best  possible  feedback  system  if  it  is 
to  achieve  its  mission  with  the  desired  qualitative  standards. 

The  significance  and  magnitude  of  the  requirements  of  field  feedback  in 
the  mission  accomplishment  of  an  Army  service  school  as  has  been  dis¬ 
cussed  demand  more  than  is  forthcoming  from  the  conventional  field 
feedback  system.  This  school  currently  uses  the  conventional  approach 
of  mailed  questionnaires  to  graduates  and  immediate  supervisors.  The 
local  experience  reinforces  the  thesis  that  the  mailed  questionnaire  has 
seriove  limitations.  Among  these  limitations  (Slide  12)  are  the  difficulty 
in  getting  a  return  of  greater  than  30%,  the  coverage  restrictions  associ¬ 
ated  with  a  questioiuiaire,  the  stigma  associated  with  such  a  device  by  the 
recipient,  and  the  lack  of  timely  responsiveness  which  is  of  prime 
importance  in  today's  school  environment.  In  short,  our  current  field 
feedback  effort  is  not  meeting  the  needs  which  have  been  outlined. 


As  a  solution  to  the  inadequacy  of  our  current  field  feedback  effort,  we 
have  developed  an  expanded  field  feedback  survey  system  which  enlarges 
the  scope  of  the  data  collection  activity,  the  sum  total  of  which  will  meet 
the  si^  lificant  needs  for  such  data.  CXir  total  system  encompasses  (Slide  13) 

-  The  field  visitation  program 

-  Research  feedback 

-  The  debriefing  of  student  personnel 

-  The  mailed  graduate  questioxmaire 

-  The  MOS  proficiency  test  data 

-  The  debriefing  of  assigned  personnel 

These  system  components  represent,  in  the  main,  known  feedback  sources 
in  a  training  environment.  The  one  which  we  have  labeled  as  research 
feedback  requires  definition  so  that  it  may  be  understood.  Research  feed¬ 
back  infers  the  local  usage  of  feedback  data  of  other  service  schools  having 
significant  blocks  of  engineer- related  instruction  in  their  curriculum.  For 
example,  the  13  hours  of  instruction  in  the  Infantry  Officer  Advanced  Course 
covering  engineer  subject  matter  and  taught  by  engineer  officers  is  of  prime 
interest  to  the  proponent  school  --  the  Engineer  School.  The  analysis  of 
appropriate  feedback  data  from  the  Infantry  School  will  provide  direction 
for  instructional  material  supporting  the  engineer  instruction  at  that  school. 

As  previously  stated,  the  major  thrust  of  this  briefing  is  the  component 
identified  as  the  field  visitation  program,  which  will  be  addressed  in  the 
remainder  of  my  remarks. 

Our  proposed  field  visitation  program  has  7  objectives.  These  objectives 
are:  (Slide  14)* 

-  To  initially  validate  task  inventories  for  all  MOS's  and  duty  positions 
impacting  upon  the  USAES  mission  and  to  provide  input  for  the  continual 
maintenance  of  these  task  inventories. 

-  To  determine  the  most  cost  effective  training  approaches  and  environ¬ 
ments  to  meet  career  development  objectives. 

-  To  determine  the  on-job  proficiency  of  school  graduates  in  the  per¬ 
formance  of  MOS  or  duty  position  tasks. 
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-  To  assess  the  learning  effectiveness  of  methods /media  used  in 
school  courses  when  measured  against  job  performance. 

-  To  obtain  reactions  to  published  trainii^  literature  for  the  purpose 
of  improving  its  utility  to  the  user. 

-  To  obtain  user  reaction  to  locally  developed  unit  training  programs 
and  to  obtain  user  recommendations  that  will  increase  the  dynamic  quality 
of  unit  training. 

To  assess  the  learning  effectiveness  of  the  content  and  approaches 
used  in  school  correspondence  courses  when  measured  against  job 
performance. 

Having  identified  the  field  visitation  program  objectives,  attention  is  now 
focused  on  three  essential  elements  of  the  field  visitation  program  (Slide  15). 
These  elements  consist  of  the  program  sample;  The  team,  its  composition 
and  mode  of  operation;  and  the  data  collection  devices.  (Slide  Off). 

When  the  element  of  the  program  sample  was  addressed,  the  answers  to 
three  questions  were  of  prime  concern.  (Slide  16)  Who  is  to  be  included 
in  the  sample?  How  large  should  the  sample  be?  Where  is  the  sample 
population  located? 

In  response  to  the  first  question,  "Who  should  be  included  in  the  sample?  ", 
the  array  of  MOS’s  for  which  the  Corps  of  Engineers  has  proponency  was 
examined  to  establish  a  priority  of  need.  This  study  identified  (Slide  17) 

16  resident  USAES  MOS  courses  and  all  20  ATC  courses  taught  at  Fort 
Leonard  Wood  for  prime  consideration.  As  the  sample  became  more 
clearly  defined  an  opportxuiity  for  gathering  data  on  a  more  limited  scale, 
but  for  a  far  greater  array  of  MOS  and  courses  appeared  feasible.  By 
relying  on  a  ^'questionnaire  only"  approach,  quantitative  data  needs  will  be 
satisfied  for  the  entire  range  of  school  proponent  courses  not  included  in 
the  base  sample.  For  planning  purposes  however,  the  original  base  of  36 
MOS  courses  (Slide  18)  was  used  for  all  calculations. 

The  questions  of  sample  size  and  geographical  location  were  resolved  with 
the  aid  of  the  most  recent  Army  authorization  documents  system  listing. 

The  worldwide  distribution  of  Engineer  personnel  was  s  creened  to  identify 
those  units  which  held  the  highest  concentrations  of  the  base  sample  of  36 
MOS's.  Of  the  several  hundred  tinits  identified,  further  screening  reduced 
the  list  (Slide  19)  to  its  final  configuration  of  57  units  situated  in  seven  major 
geographical  locations.  The  screening  procedures  used  assured  not  only  the 
highest  concentrations  of  individual  MOS  within  units  but  also  the  highest 


concentration  of  clusters  of  MOS,  Thus  any  adjustment  of  the  unit  list 
would  result  in  a  loss  of  efficiency  in  terms  of  both  time  and  data  collection 
effort.  As  a  result  of  the  screening  process,  the  incumbent  population  for 
the  36  MOS*s  was  reduced  (Slide  20)  from  29, 115  to  9,  137  enlisted  engineers. 

This  sample  of  9,  137  will  be  surveyed  by  questionnaire  in  the  unit  environ¬ 
ment  with  a  reduced  sample  of  2,  812  being  interviewed.  The  reduced  sample 
provides  for  a  maximum  of  100  per  MOS,  resulting  in  a  60%  savings  of  time 
with  only  a  negligible  loss  of  data. 

While  not  a  consideration  of  sample  size,  sample  quality  was  improved  by 
insuring  that  only  STRAF  units  were  included  in  the  CONUS  portion  of  the 
sample. 

Of  the  57  units  constituting  the  sample  19  different  unit  types  are  represented, 
(Slide  off). 

The  second  essential  element  of  the  program  concept  is  the  visitation  team 
and  its  mode  of  operation.  After  examining  the  sample  size  and  a  chool 
resources,  it  was  determined  that  a  single  team  of  5  members  could  support 
the  program.  The  logic  underlying  this  determination  is  primarily  based  on 
the  listing  of  numbers  of  personnel  per  unit  to  be  interviewed,  and  an  average 
interview  length  of  1/2  hour.  These  guides  were  extended  to  structure  the 
’’Team  Day.  ”  The  Team  Day  is  the  model  for  the  team’s  activities  during 
a  typical  8  hour  workday.  Because  questionnaire  data  is  to  provide  the 
foundation  for  the  followup  interview,  these  had  to  be  administered  early  in 
the  unit  visit  accompanied  by  a  unit  briefing.  It  is  planned  therefore,  that 
the  first  team  day  (Slide  21)  would  be  divided  into  a  two  hour  block  of  unit 
briefings  and  questionnaire  administration  followed  by  a  six  hour  block  of 
interviews.  All  subsequent  days  within  the  same  unit  would  consist  of  8 
hour  blocks  of  interviews.  This  plan  enables  the  team  to  work  at  the  rate  of 
60  interviews  for  the  first  day  and  80  interviews  on  subsequent  days.  Unit 
briefings  are  structured  to  achieve  three  purposes.  They  will  first  open  a 
two-way  communication  channel  between  the  school  and  units  in  the  field. 

The  team  will  be  equipped  to  provide  the  unit  with  the  most  current  school 
publications,  plans,  concepts,  and  materials  which  have  been  determined  de¬ 
sirable  by  prior  arrangement.  The  forging  of  this  vital  link  between  school 
and  field  is  considered  of  prime  importance.  The  second  purpose  of  the 
unit  briefings  is  to  plan  the  activities  of  the  team  during  their  visitation. 

With  the  exception  of  the  questionnaire  first,  interview  second  concept,  all 
team  activities  wijl  follow  the  wishes  of  the  unit  commander.  This  procedure 
and  the  fact  that  interview  schedules  have  been  planned  to  last  no  longer  than 
two  days  are  designed  to  minimize  the  disruption  of  unit  activities.  It  is 
planned  that  the  unit  briefings  will  be  attended  by  key  unit  personnel.  During 


these  briefings,  data  collection  devices  will  be  distributed  to  collect 
feedback  data  from  the  assigned  officers  and  NCO's.  The  data  will  be 
used  as  the  design  base  of  the  officer  and  NCO  education  system  courses. 
This  officer  and  NCO  data  collection  effort  rounds  out  the  third  objective 
of  the  unit  briefings. 

In  terms  of  composition  (Slide  22)  the  five  man  team  consists  of  a  LTC 
Team  Chief  with  either  a  Combat  or  Construction  Engineering  MOS;  a 
Warrant  Officer  with  a  maintenance  background;  and  three  Noncommissioned 
Officers  with  combat  engineering,  construction  engineering  and  topographic 
backgrounds.  While  cross -field  interviewing  is  anticipated  because  of 
variations  in  \mit  composition,  the  team  backgrounds  coupled  with  pre-visit 
training  should  preclude  any  data  collection  problems.  (Slide  off). 

The  final  element  of  the  field  visitation  program  deals  with  the  design  of 
the  data  collection  devices.  The  wide  variety  of  data  needs  represented 
by  the  program  objectives  coupled  with  the  sample  size  and  composition 
creates  an  imposing  data  collection  and  analysis  problem.  From  necessity 
a  modular  approach  to  device  design  was  employed.  Five  basic  modules, 
each  serving  a  specific  set  of  objectives  were  devised.  These  modules 
would  be  configured  into  combinations  to  serve  specific  needs.  (Slide  23). 

The  work  performance  module  is  designed  to  provide  data  for  use  of  task 
inventory  validation,  MOS  evaluation,  and  MOS  structuring.  The  format 
of  the  task  list  section  of  this  module  is  identical  to  that  used  by  the 
military  occupational  data  bank  questionnaire.  This  design  feature  will 
provide  a  means  for  cross-validation  and  access  to  the  enlisted  evaluation 
data  pool.  The  bulk  of  the  data  solicited  by  the  questionnaire  is  in  a  form 
amenable  to  machine  processing.  An  open-ended  section  provides  leads 
to  be  exploited  during  the  follow-up  interview  sections. 

The  work  proficiency  module  provides  the  capability  for  assessing  the 
quality  of  USAES  graduates  and  thereby  providing  insights  to  the  possible 
need  for  course  modification.  Both  machine  scorable  and  open-ended 
sections  are  provided.  This  questionnaire  is  to  be  completed  by  the  super¬ 
visor  of  the  MOS  holder. 

The  literature  module  is  not  unique  to  any  specific  MOS  and  thus  will  be 
administered  to  all  MOS.  The  design  is  directed  to  collecting  data  con¬ 
cerning  literature  usage  and  user  suitability. 

The  unit  training  module  centers  on  the  collection  of  information  concerned 
with  unit  training  improvement.  This  module  will  be  completed  by  all 
personnel  regardless  of  MOS.  The  design  of  this  instrument  will  supplement 
and  extend  data  collected  earlier,  by  mail,  as  part  of  the  Engineer  School 
Dynamic  Training  Council  Questionnaire  Survey  Program. 


The  nonresident  training  module  will  be  completed  by  or2y  those  personnel 
who  have  availed  themselves  of  this  mode  of  instruction.  The  collected 
data  will  provide  insights  to  possible  course  extension  and  improvement. 

By  including  common  identification  features  in  each  of  the  modules^  the 
data,  after  collection,  may  be  analyzed  either  in  its  entirety  or  separately 
by  the  individual  requestors.  Standardized  formating  permits  maximum 
machine  processing  for  bulk  data  while  the  unstructured  open-ended  sections 
permit  entries  for  interview  structuring.  Modular  design  also  permits  data 
separation  by  program  objectives. 

An  overview  (Slide  24)  of  the  field  visitation  program  illustrates  how  the 
components  function  to  accomplish  their  purpose.  This  overview  addresses 
the  first  objective  of  the  program  which  relates  to  the  validation  and  main¬ 
tenance  of  task  inventories.  The  field  visitation  program’s  work  performance 
and  work  proficiency  modules  are  structured  around  task  inventories.  The 
work  performance  module  is  administered  to  the  job  holder  to  determine 
which  tasks  he  performs  and  whether  he  considers  the  tasks  difficult.  The 
follow-up  interview  would  determine  the  nature  and  details  of  any  task 
difficulty.  The  work  proficiency  module  is  structured  on  duty  areas  which 
are  groupings  of  tasks.  This  module  is  administered  to  the  job  holder’s 
supervisor,  and  evaluates  the  competency  of  the  job  holder’s  task  perfor¬ 
mance.  Both  modules  contain  a  device  for  matching  job  holder  and  super¬ 
visor  questionnaires.  Ajialysis  of  the  data  will  determine  the  percentage 
of  the  sample  who  perform  the  task»  which  tasks  are  considered  difficult, 
the  reason  for  any  difficulty,  and  an  evaluation  of  the  proficiency  of  task 
performance.  The  findings  would  determine  the  need  for  school  training 
for  a  particular  task,  the  adequacy  of  present  training  to  satisfy  field  needs, 
the  variance  of  job  performance  by  unit  type  or  geography,  and  the  impact  of 
time  on  the  job  upon  task  performance.  In  addition,  the  findings  will  provide 
the  answers  to  many  other  school  mission  questions. 

This  briefing  has  (Slide  25): 

-  Enumerated  the  feedback  needs  in  an  Army  service  school  environment 

>  Described  the  current  feedback  posture  in  this  environment 
Identified  the  totality  of  the  USAJES  field  feedback  survey  system 
rtbed  the  objectives  of  the  proposed  field  visitation  program 
*  the  basic  elements  of  the  field  visitation  program  (Slide  Off). 
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-  TO  EXAMINE  THE  BASIC  ELEMENTS  OF  THE  FIELD 
VISTATION  PROGRAM 
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FIELD  VISITATION  PROGRAM 
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USE  OF  TELEVISION  AND  VIDEOTAPES  IN  THE  TRAINING 
OF  POLYGRAPH  EXAMINERS  AT  USAMPS 


Dr.  Neal  B.  Andregg 
Educational  Advisor,  USAMPS 

The  United  States  Army  Military  Police  School  has  found  that  the  use 
of  television  and  videotapes  contributes  greatly  in  motivating  students 
in  our  Polygraph  Examiner  Training  Course.  We  present  a  series  of 
situations  in  which  one  or  more  of  the  subjects  being  interrogated  will 
use  ego  defense  mechanisms  to  preserve  their  feelings  of  adequacy 
and  personal  worth.  These  mechanisms  such  as  rationalization,  identi¬ 
fication,  projection  emd  displacement,  are  called  into  play  whenever 
subjects  find  themselves  in  situations  in  which  a  threat  to  the  totegrity 
of  self  is  present.  In  each  skit  or  situation,  students  are  asked  to 
identify  the  ego  defense  mechanism  which  the.  subject  used  and  any 
remarks  which  were  made  that  helped  them  to  identify  it.  They  are 
also  asked  to  point  out  any  mistakes  which  were  made  in  the  interroga- 
*^1'^  n  and  what  actions  they  would  have  taken  in  an  attempt  to  solve  the 
c«.se.  Videotapes  can  be  used  for  both  class  discussion  and  for  testing. 
The  videocassette  which  I  shall  use  in  showing  you  portions  of  a  video¬ 
tape  which  I  produced  is  easy  to  use.  An  instructor  can  use  it  any  time 
he  desires  to  stimulate  class  discussion  or  he  can  employ  it  to  test 
students  in  many  soft  skills  areas. 

(Show  ten  minute  videocassette) 

Script  of  video  tape:  (Audio  portion  only) 


USAMPS  Presents 
DYNAMICS  OF  HUMAN  BEHAVIOR 
EGO  DEFENSE  MECHANISMS 
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SCENE  #1 


Mr.  Raciborski 

Hi»  Mrs.  Wdllis,  you  certainly  are  a  difficult  person  to  keep  in  touch 
with.  This  is  the  third  time  this  week  that  1  have  tried  to  see  you. 

Mrs.  Willis 

Oh,  I'm  sorry.  I  guess  1  just  forgot  the  other  two  appointments.  When 
did  you  try  to  see  me? 

Mr.  Raciborski 

Let  me  see  now,  the  first  time  was  last  Monday;  the  second,  I  believe, 
was  the  day  before  yesterday.  Yes,  Wednesday,  in  the  afternoon. 

Mrs.  Willis 

Oh,  I  remember  now,  when  we  were  really  busy  at  the  office  with  exams 
last  Monday,  and  Wednesday  I  had  an  appointment  to  get  my  hair  fixed. 

1  just  forgot. 

Mr.  Raciborski 

You  certainly  do  look  nice  today.  By  the  way,  tell  me,  did  you  go  to  the 
beauty  shop  yesterday  afternoon? 

Mrs.  Willis 

Yes,  I  did.  My  boyfriend,  he  likes  for  me  to  look  nice. 

«  * 

Mr.  Raciborski 

That's  understandable.  Tell  me  now,  Betty,  considering  what  your  income 
is,  how  do  you  afford  all  your  expensive  clothes,  jewelry,  and  daily  hair 
set?  . 

Mrs.  Willis 

That's  none  of  your  business.  I  don't  have  to  talk  to  you  about  that. 

Haven't  you  heard  of  the  5th  Amendment? 

Or.  Andregg 

What  ego  defense  mechanism  did  our  subject  use  in  this  situation? 

(Test  Item  Card) 
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SCENE  §?. 


Dr.  Andregg 

The  investigator  has  informed  the  subject  of  her  rights  and  proceeds  with 
the  interrogation.  He  thinks  that  he  has  finally  persuaded  her  to  talk. 
However,  I  will  let  you  be  the  judge  as  to  how  successful  he  is.  Please 
continue  to  watch  for  any  ego  defense  mechanism. 


Mr.  Racibovski 

Now,  Betty,  let's  consider  your  expensltve  clothes  and  jewelry  again. 

You  don't  really  expect  me  to  believe  that  you  get  all  that  on  your  pay. 

Mrs.  Willis 

I  could  care  less  what  you  think.  1  don't  remember  where  thay  all  came 
from.  A  girl  does  receive  gifts  you  know. 

Mr.  Raciborski 

Tell  me,  where  did  you  get  that  large  diamond? 

Mrs.  Willis 

One  of  my  admirers.  ' 

Mr.  Raciborski 
Which  one? 

Mrs.  Willis 
1  don't  remember. 

Mr.  Raciborski 

You  don't  remember.  You're  smart  enough  to  realiae  that  I  couldn't 
believe  that  if  my  life  depended  on  it.  Don't  you  realise  that  you  could 
get  into  a  lot  of  trouble  receiving  stolen  property? 

Mrs.  Willis 

I'll  have  you  know  that  the  men  I  go  out  with  can  well  afford  to  give  me 
nice  presents.  They  don't  have  to  resort  to  stealing.  Some  of  them  even 
take  me  to  the  Pinnacle  Club  and  that's  only  open  to  men  of  distinction. 

Dr.  Andregg 

Did  our  subject  use  an  ego  defense  mechanism?  If  so,  which  one  did 
she  use? 


(Test  Item  Card) 


SCENE  *3 


Dr.  Andregg 

The  invaatigetor  continuea  with  hia  interrogation  and  ia  peraiatent  in  hia 
efforta  to  get  the  aubject  to  talk. 


Mr.  Raciboraki 

Betty,  juat  take  a  look  at  the  aiae  of  that  diamond.  You  don't  really 
expect  me  to  believe  that  you  don't  remember  who  gave  you  a  diamond 
that  aiae,  do  you? 

Mra.  Willia 

Oh,  not  really.  How  could  I  forget  aomeone  who  haa  given  me  ao  many 
nice  thinga. 

Mr.  Raciboraki 

What*  a  hia  name,  Betty? 

Mra.  WUlia 

(PAUSE)  1  am  not  telling  you  hia  name.  He' a  been  too  nice  to  me,  and 
I'm  not  getting  him  into  any  trouble. 

Mr.  Raciboraki 

Betty,  aa  you  know,  my  inveatigation  indicatea  that  diamond  ia  atolen 
property,  and  moat  likely,  the  other  diamonda  probably  are  alao.  Now 
again,  how  and  where  did  you  get  them? 

Mra.  Willis 

Well  soveral  years  ago  this  real  nice  man  that  I  knew,  he  wanted  me 
to  try  them  on  and  1  did,  and  he  wanted  me  t'o  keep  them  and  1  did.  A  girl 
would  be  craay  to  refuse. 

Mr.  Raciboraki 

Are  you  sure  now  that  it  was  several  years  ago  and  maybe  not  just  several 
weeks  ago? 

Mrs.  Willie 

1  don't  pay  any  attention  to  time.  He  just  insisted  that  I  keep  them. 

Mr.  Raciboraki 

Betty,  I  want  his  name,  and  I  want  it  now. 


Mr*.  Willi* 

Well»  I'm  not  going  to  tell  you  hi*  name.  He'*  been  too  nice  to  me^  and 
I'm  not  going  to  equeal  on  him. 

Mr  Raciboreki 

Betty,  think  of  thi*  a  minute.  I  know  the  girl  who  lo*t  those  diamond*. 
She  i*  a  very  sensitive  person.  She  ha*  attached  a  great  deal  of  senti¬ 
mental  value  to  them.  In  fact,  I  would  bet  right  now  she's  crying  her 
eyes  out.  Now,  you  wouldn't  want  to  wear  those  diamonds  if  you  knew, 
by  so  doing,  you  would  be  making  someone  miserable,  would  you? 

Mrs.  Willis 

No,  I  wouldn't,  but  neither  would  I  want  to  get  someone  in  trouble  that's 
been  so  nice  to  me  and  given  me  so  nutny  nice  things  and  always  wanted 
the  very  best  for  me. 

Mr.  Raciborski 

Well,  Betty,  I  guess  we  might  as  well  stop  wasting  time.  I'll  just  have 
to  find  out  who  he  is  without  your  help.  For  now  though,  I'm  going  to 
need  those  diamonds  for  the  purpose  of  making  a  positive  identification. 

Mrs.  Willis 

(PAUSE)  Well,  you  can  have  them,  but  you'll  have  to  give  me  a  receipt, 
and  I'm  sure  that  you'll  find  they  are  not  stolen  as  1  have  been  told  they 
are  insured  for  severed  thousand  doUare. 

Dr.  Andregg 

What  ego  defense  mechanism  did  our  subject  use? 

(Test  Item  Card) 

Dr.  Andregg 

What  remarks  did  she  madce  that  led  you  to  select  this  ego  defease 
mechanism? 

(Test  Item  Card) 


SCENE  #4 
Dr.  Andregg 

The  subject  has  just  returned  from  the  interrogation  conducted  by  the 
criminal  investigator  and  is  talking  with  her  co-workers.  Be  alert  for 
any  ego  defense  mechanism  which  she  may  use. 


Mrs.  Sisemore 

S«y,  Betty,  you  reelly  do  look  bushed.  You  look  like  you're  been  through 
the  mill.  How  about  me  getting  you  a  coke? 


Mrs.  Willis 

Yes,  I  think  I  need  one  after  that  grilling  that  I've  been  through.  I  have 
never  been  asked  so  many  embarrassing  questions  in  such  a  short  period 
of  time. 

Mrs.  Sisemore 

(Gives  coke  to  Mrs.  Willis)  Maybe  this  will  make  you  feel  better. 

Mr^.  Willis 

Oh,  thanks.  Thanks.  This  really  hits  the  spot.  Maybe  it  will  help  me 
forget  that  rude  investigator  who  asked  me  all  those  personal  questions. 

Mrs.  Prince 

(Walks  into  the  scene)  Oh,  is  somebody  asking  you  personal  questions? 
Mrs.  Willis 

What  if  they  were.  It's  no  concern  of  yours.  You're  as  nosy  as  some  of 
those  old  biddies  at  the  beauty  shop. 

Dr.  Andregg 

What  ego  defense  mechanism,  if  any,  was  used  in  this  situation? 

(Test  Item  Card) 


SCENE  #5. 

Dr.  Andregg 

The  subject  talks  with  her  boyfriend  the  day  following  the  interrogation. 
Remember  that  both  individuals  may  use  ego  defense  mechanisms  in  this 
situation. 

Mrs.  Willis 

Honey,  am  I  glad  to  see  you.  That  investigator  asked  me  some  really 
embarrassing  questions,  but  I  didn't  implicate  you. 

Mr.  Moree 

That's  real  nice  of  you,  and  I  appreciate  it.  As  a  matter  of  fact,  this 
evening.  I'll  show  you  just  how  much  I  do  appreciate  it.  What  did  you  do, 
clam  up  completely? 
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Mrs.  Wmia 

W«U»  1  did  tell  him  a  little  bit  when  he  inferred  that  yen  were  a  common 
thief.  He  insulted  you. 

Mr.  Moree 

Just  what  did  you  tell  him? 

Mrs.  Willis 

Well,  I  told  him  that  the  men  I  go  out  with  could  afford  to  give  me  nice 
gifts  and  ta^e  me  to  nice  places,  they  don't  have  to  resort  to  stealing. 

Mr.  Moree  ‘ 

Did  you  give  him  any  names  of  places? 

Mrs.  WUlis 

Not  at  first,  but  he  kept  bugging  me  and  making  me  nervous,  so  1  finally 
mentioned  the  Pinnacle  Club. 

Mr.  Moree 

Well,  that's  great,  that  will  throw  him  off  completely.  We  just  won't  go 
back  there  anymore.  As  a  matter  of  fact,  we  always  have  more  fiin  at 
the  other  nightclubs  anyway. 

Dr.  Andregg 

What  ego  defense  mechanisms  were  used  in  this  scene,  and  by  whom  were 
they  used? 

(Test  Item  Card) 

Dr.  Andrej;g 

How  would  you  use  your  knowledge  of  human  behavior  and  ego  defense 
mechanisms  in  an  attempt  to  break  the  case? 

(Test  Item  Card) 

USAMPS  has  also  developed  similar  videotapes  in  other  soft  skills  areas 
and  use  television  cameras  and  playback  equipment  to  record  and  critique 
instructor's  presentations  and  counseling  and  interrogation  techniques  of 
students.  We  have  found  theaS' methods  most  effective  in  improving  these 
soft  skills,  in  developing  and  maintaining  student  interest,  and  in  motivating 
our  students  to  learn  more  effectively. 
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COLONEL  G.  B.  HOWARD 


Colonel  Howard  is  currently  the  Product  Manager  for  CONARC's 
Computerized  Training  System,  having  come  to  that  assignment  from 
Germany  where  he  was  Connmanding  Officer  of  the  4th  Signal  Service 
Group  and  Community  Leader  for  the  Karlsruhe  area.  Prior  to  that, 
he  served  two  years  of  duty  with  the  U.  S.  Delegation  to  the  NATO 
Military  Committee  in  Brussels,  Belgium. 

Colonel  Howard  graduated  from  The  Citadel  in  1945,  and  entered  the 
Army  as  a  Private  in  October  of  that  year.  After  a  year  of  enlisted 
service,  he  was  given  a  direct  commission  as  a  2d  Lieutenant  in  the 
Army  of  the  United  States. 

Colonel  Howard  has  served  on  the  staff  and  faculty  of  the  Army  Security 
Agency  School,  the  Signal  School,  the  Command  and  General  Staff  College, 
and  the  Army  Artillery  School.  His  military  schooling  includes  enlisted 
attendance  at  the  Army  Security  Agency  School,  the  Signal  Officers 
Advance  Course,  a  Spanish  course  at  the  Army  Language  School,  the 
Command  and  General  Staff  College,  and  the  Army  War  College. 

His  other  assignments  include  ASA  detachment  commander  in  Trieste, 
Signal  Company  Commander  during  the  Korean  conflict,  four  years  as 
Plans  Officer  and  Nuclear  Weapons  Officer  in  the  5th  Division  Artillery, 
two  years  of  advisory  duty  in  Ecuador,  and  a  tour  with  the  Headquarters 
of  US  Army  Vietnam. 

His  awards  and  decorations  include  the  Legion  of  Merit,  the  Meritorious 
Service  Medal,  and  the  Army  Commendation  Medal  with  Oakleaf  Cluster. 
An  Eagle  Scout  himself.  Colonel  Howard  has  served  thirty-two  years 
with  boy  scouting  and  is  recipient  of  the  Silver  Beaver  Award.  He  is  also 
in  the  Sxmday  School  program,  having  served  as  superintendent  or  teacher 
on  each  of  his  assignments.  His  other  interests  include  tennis  and  bowling. 


project  abacus 

A  Prototype  Computerized  Training  System  for  the  Army 

Colonel  G.  B.  Howard 
Product  Manager,  USASCS 


INTRODUCTION 

The  purpose  of  Project  ABACUS  briefly  stated  is  to  develop,  test  and 
evaluate  a  large-scale  prototype  Computerized  Training  System  (CTS), 
which  combines  the  application  of  Computer  Assisted  Instruction  (CAI) 
and  Computer  Managed  Instruction  (CMI).  The  project  will  cover  the 
four  year  period  b^^ginning  1  August  1972  and  ending  1  August  1976.  It 
is  under  the  direction  of  the  Office  of  the  Product  Manager  (PM),  located 
at  the  U.  S.  Arruf  Signal  Center  and  School,  Fort  Monmouth,  New  Jersey. 

BACKGROUND 

A  six-month  study  coiniducted  by  the  Director  of  Management  Information 
Systems,  Department  of  the  Army,  resulted  in  the  establishment  by 
CONARC  i  f  a  Computer  Assisted  Instruction  (CAI)  Task  Group  in  November 
1971.  The  group  was  chaired  by  a  representative  of  the  U,  S.  Army 
Signal  Center  and  School  and  included  participating  members  from  ten 
other  CONARC  schools.  The  final  report,  submitted  in  April  1972, 
recommended  initiation  of  the  prototype  to  include  integration  of  CMI-CAI, 
use  of  a  multiprocessor  mini-computer  concept  for  the  central  system, 
and  use  of  a  variety  of  course  types  for  evaluation.  Although  directed  to 
concentrate  on  technical  training,  the  group  noted  in  its  findings  that  soft- 
skill  courses  might  be  even  more  cost  effective  for  CAI.  An  outgrowth  of 
the  group's  recommendations  was  the  establishment  of  Project  ABACUS. 
The  USASCS  was  selected  for  the  site  for  the  Product  Manager's  Office 
because  of  the  existing  expertise  at  that  location. 

CAI  AT  USASCS 

The  CAI  Project  at  USASCS  originated  in  August  1966  for  the  purpose  of 
investigating  the  feasibility  of  employing  CAI  in  conducting  Common  Basic 
Electronics  Training.  Using  an  IBM  1500  Instructional  System,  the  pro¬ 
ject  conducted  a  training  and  evaluation  program  encompassing  four  years, 
during  which  more  than  1300  students  underwent  training.  The  technical 
reports  and  papers  resulting  from  this  effort  are  considered  among  the 
most  authoritative  in  the  entire  field  of  CAI.  As  a  part  of  the  evaluation, 
a  closely  controlled  experiment  was  conducted  to  compare  CAI  with 
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convention  classroom  instruction.  The  results  are  shown  in  Figure  #1. 
From  this  data  it  was  concluded  that  CAl  was  not  only  feasible  from  a 
training  effectiveness  standpoint,  but  could  bring  about  significant  savings 
in  training  time. 

PROJECT  SCHEDULE 

The  scope  of  the  CTS  Project  includes  the  design,  development,  imple¬ 
mentation,  operation  and  evaluation  of  the  integrated  prototype  CTS. 

The  project  will  be  carried  out  in  five  separate  phases:  (See  Figure  #2). 

1.  System  Design.  The  Human  Research  Resources  Organization 
(HumRRO)  is  under  contract  for  design  with  direction  and  guidance  from 
the  PM  through  the  Army  Research  Office  (ARO).  This  phase  nins  through 
the  first  ten  months  of  the  project  which  began  1  August  1972  and  involves 
the  specific  design  of  a  complete  system  for  use  by  the  Prototype  CTS. 

2.  System  Development.  Based  on  specifications  determined  in 
Phase  1,  a  contractor  will  be  selected  by  the  middle  of  next  year.  The 
system  development  contractors  will  be  responsible  for  the  integration  of 
the  hardware /software  into  an  operational  system. 

3.  Course  Development.  The  PM  is  responsible  for  development  of 
the  course  material  for  implementation  of  the  prototype  CTS.  The  Office 
of  the  PM  has  a  head  start  on  this  phase.  Student  terminals  connected 

to  the  PLATO  IV  System  at  the  University  of  Illinois  will  be  used  to  train 
personnel  as  instructional  programmers  and  for  initial  development  of 
the  course  material  designated  for  the  Prototype  CTS. 

4.  CTS  Operation.  The  prototype  system  will  operate  a  minimum  of 
one  year  prior  to  procurement  of  operational  systems.  This  phase  is 
scheduled  to  begin  in  January  1975  and  run  through  April  1976. 

5.  CTS  Evaluation.  Evaluation  will  be  conducted  by  the  PM  con¬ 
currently  with  Phase  IV,  and  will  address  the  feasibility  and  effective¬ 
ness  of  the  entire  system.  It  will  conclude  in  August  1976.  Contractor 
support  will  be  used  as  appropriate. 

ORGANIZATION 

The  relationship  of  the  Product  Manager  to  other  agencies  is  given  in 
Figure  #3.  It  will  be  noted  that  the  command  line  is  direct  to  CONARC, 
with  monitorship  provided  by  USASCS.  All  activities  are  conducted  under 


a  charter  similar  to  that  covering  project  managership  within  the  Army 
Materiel  Command.  The  Steering  Advisory  Group  (SAG)  has  the  responsi- 
bilit>  for  monitoring  the  CTS  project  for  the  Department  of  the  Army. 

The  SAG  meets  monthly  and  provides  a  mechanism  for  coordination, 
exchange  of  information,  and  review  of  the  Prototype  CTS  project  by  all 
interested  parties.  The  SAG  also  exercises  control  over  the  functional 
requirements  and  technical  characteristics  of  the  Prototype  CTS.  The 
Chairman  of  this  group  is  the  Director  of  Army  Research,  the  Product 
Manager  of  the  CTS  is  the  Executive  Secretary  of  the  SAG,  and  the  follow¬ 
ing  agencies  supply  membership  on  the  SAG:  DA,  DMIS;  DA,  DCSPER; 
OCRD;  USCONARC;  USASCS;  and  the  Office  of  the  Product  Manager. 

Internally  the  Office  of  the  Product  Manager  is  organized  as  shown  in 
Figure  #4.  Additional  personnel  will  be  required  as  the  project  progresses, 
particularly  in  Course  Development  and  Systems  Operations.  The  Evalua¬ 
tion  Division  will  remain  small  with  support  being  provided  through  con¬ 
tractors/consultants  as  required. 

HARDWARE 

As  mentioned  earlier,  the  hardware  will  be  based  on  the  multi-processor 
mini-computer  concept.  The  interrelationship  of  the  major  components 
for  such  a  system  are  shown  in  Figure  #5. 

Major  design  considerations  or  characteristics  have  been  established  for 
CTS  so  that  it  can  function  effectively  and  efficiently  in  providing  a  group 
of  students  with  a  highly  adaptive,  self-paced  training  environment.  These 
are  identified  and  briefly  discussed  as  follows: 

Number  of  Terminals.  As  presently  conceived  the  Initial  version  of 
the  system  will  contain  128  terminals  and  have  the  capability  to  expand  to 
about  1000  terminals. 

Subject  Matter  Storage.  The  system  will  have  the  capability  to  store 
subject  matter  equivalent  to  about  900  hours  of  conventionally  presented 
instruction. 

Response  Time.  The  system  will  process  an  entry  made  by  a  student 
and  return  appropriate  information  to  the  student  within  a  time  delay  not 
exceeding  four  seconds.  It  is  expected  that  the  system  response  time  will 
average  about  2-3  seconds. 

Remote  Terminal  Operation.  The  system  will  be  capable  of  servicing 
terminals  distributed  in  a  typical  military  training  complex.  The  maximum 
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distance  between  a  terminal  and  the  computer  is  established  at  about 
five  miles.  This  capability  will  be  attained  without  resorting  to  special 
interfacing. 

On-Line  Entry  of  Course  Material.  All  course  material  as  it  is 
developed  will  be  entered  directly  into  the  system  from  a  terminal. 

Punched  card  entiry  will  be  limited  to  specialized  activities  involving  the 
operation  of  the  system  and  batch  processing. 

Simultaneous  Operations.  The  system  will  enable  simultaneous 
activity  by  students  and  instructional  programmers.  This  means  the 
course  authoring  and  debugging  can  take  place  while  students  are  receiving 
instruction.  The  requirements  for  the  inclusion  of  batch  processing  among 
the  simultaneous  operations  is  not  firmly  established  at  this  time.  This 
activity  can  take  place  during  the  third  shift. 

Instructional  Model.  The  decision  making  process  incorporated  in  the 
instructional  model  will  be  attentive  to  the  latest  advances  in  education 
and  training  technology.  However,  the  level  of  sophistication  will  be 
initially  limited  to  techniques  and  strategies  of  proven  merit. 

Authoring  Language.  The  language  will  consist  of  the  necessary 
commands  to  enable  the  computer  system  to  present  subject  matter  in 
the  context  of  the  instructional  model.  These  commands  will  be  designed 
to  require  a  minimal  coding  effort  for  course  materials  development. 

In  addition,  the  langtiage  will  contain  the  necessary  commands  to  simplify 
on-line  authoring  and  debugging. 

The  terminal  configuration  shown  in  Figure  #6  is  not  necessarily  the 
one  that  will  be  used  in  CTS,  but  is  presented  as  a  possible  configuration 
that  will  provide  a  very  effective  learning  environment.  From  the  stu¬ 
dent’s  point  of  view  this  environment  is  the  most  important  part  of  the 
system  because,  after  all,  it  is  here  that  learning  takes  place.  The 
multi-media  aspect  of  the  terminal  configuration  is  readily  apparent. 
Considering  the  fact  that  the  student  will  spend  a  considerable  part  of 
each  day  in  this  environment,  the  multi-media  approach  will  provide 
sufficient  variety  in  activity  to  lessen  the  effects  of  fatigue  or  boredom 
that  would  normally  accrue  from  extended  exposure  to  a  single  medium 
or  activity.  Furthermore,  the  multi-media  configuration  will  enable  the 
instructional  programmer  to  utilize  a  mix  of  media  that  is  most  appro¬ 
priate  for  developing  course  materials  geared  to  the  attainment  of  specific 
training  objectives.  The  devices  shown  are  described  as  follows: 

1.  Primary  Display.  This  device  is  essentially  a  TV  receiver.  It 
can  be  connected  directly  to  the  central  computer  system  or  to  the  closed 
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circuit  television  system.  Most  of  the  time  the  primary  display  device 
will  be  connected  to  the  central  computer  system  for  self-paced  interactive 
instr  iction.  Group  instruction  or  briefings  can  be  provided  by  switching 
the  device  to  the  closed  circuit  television  system.  Typical  examples  of 
this  utilization  are  command  briefings,  safety  and  security  lectures,  and 
special  announcements.  In  addition  the  video  cassette  can  be  utilized  to 
provide  supplementary  instruction.  The  audio  capability  required  for 
closed  circuit  TV  and  video  cassette  modes  of  utilization  can  also  be 
employed  when  the  primary  display  is  connected  to  the  central  system. 
Display  of  textual  material  by  the  primary  device  will  be  free  of  flicker 
and  consist  of  characters  no  less  than  1/4’*  in  size.  The  total  text  handling 
capability  will  be  about  600  individual  characters.  The  primary  display 
will  also  have  a  graphics  capability  involving  simple  line  drawings  such  as 
schematics  and  pictorial  representations. 

2.  Keyboard.  The  keyboard  will  provide  the  student  with  a  means 
of  answering  questions  or  requesting  information.  It  will  consist  of  a 
standard  typewriter  keyset  with  additional  special  function  keys  to  simplify 
operation  of  the  terminal.  For  example,  a  NEXT  key  will  be  provided  so 
that  the  student  can  indicate  to  the  computer  that  an  answer  has  been  com¬ 
pleted  or  a  request  has  been  made. 

3.  Secondary  Display.  This  part  of  the  terminal  configuration  is 
used  to  provide  illustrations  to  supplement  the  textual  instruction  presented 
on  the  primary  display.  Generally  speaking,  the  illustrations  are  of  a 
higher  order  of  complexity  or  detail  than  can  be  presented  on  the  primary 
display  when  it  is  connected  to  the  central  system.  In  addition,  the 
secondary  display  is  used  when  a  sufficient  area  is  not  available  on  the 
primary  display  for  simultaneous  display  of  text  and  graphics.  The  device 
is  normally  a  rear  screen  filmstrip  or  micro-fiche  projector.  Synchroni¬ 
zation  with  the  textual  material  on  the  primary  display  is  obtained  by  giving 
each  frame  a  code.  Thus,  the  instructional  programmer  can  call  up  a 
specific  frame  to  supplement  textual  material. 

4.  Touch  Panel.  This  device  adds  a  new  dimension  to  the  multi- 
media  capability  of  the  terminal  because  it  can  be  used  to  overlay  any 
display  whether  or  not  the  display  is  connected  to  the  central  system. 

For  example,  it  can  overlay  a  schematic  diagram  used  for  teaching  circuit 
analysis  or  troubleshooting.  Its  operation  is  very  simple  because  the 
student  need  only  touch  his  finger  to  the  point  of  interest  which  is  auto¬ 
matically  identified  by  the  computer.  Thus,  the  student  can  devote  his 
full  attention  to  the  display  without  the  distractions  of  light  pen  or  keyboard 
operation. 
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5.  Training  Equipment  and  Device8.  The  prese  nce  of  training 
equipment  and  devices  within  the  terminal  configuration  attests  to  the 
attention  given  to  the  "hands-on"  approach  to  training  advocated  by 
CONARC,  Experience  at  USASCS  has  conclusively  demonstrated  that 
students  can  be  taught  practical  skills  by  means  of  computer  assisted 
instruction  and  that  the  varied  activity  inherent  to  this  approach  main¬ 
tains  a  high  level  of  student  interest  and  motivation. 

COURSE  DEVELOPMENT 

Instructional  materials  developed  for  CTS  will  comply  with  CONARC 
training  policy.  This  means  that  the  instruction  will  reflect  the  systems 
engineering  approach  to  training  with  great  emphasis  placed  on  the  develop¬ 
ment  of  job  skills  and  knowledges.  Furthermore,  care  will  be  taken  to 
assure  that  the  training  methods  and  media  used  within  CTS  are  appro¬ 
priate  for  the  attainment  of  specific  training  objectives  and  not  because  of 
"gimmick"  appeal.  Generally  speaking,  the  course  material  will  be 
developed  for  presentation  on-line  in  the  terminal  configuration  and  off¬ 
line  in  practical  exercise  labs.  The  on-line  course  material  will  employ 
CAI  modes  such  as  tutorial,  drill  and  practice,  and  dialogue.  The  off¬ 
line  instruction  will  be  computer  managed  to  take  full  advantage  of  the 
diagnostic  and  record  keeping  capabilities  of  CTS. 

The  steps  involved  in  developing  a  CAI  course  of  instruction  are 
given  in  Figure  #7.  It  will  be  noted  that,  with  the  exception  of  the  on-line 
review,  testing  and  debugging,  these  are  the  same  as  for  any  other  systems 
engineered  course. 

As  shown  in  Figure  #8,  each  lesson  module  consists  of  a  pretest, 
lesson  element,  and  a  lesson  review.  The  terminal  performance  objectives 
with  their  associated  enabling  objectives  are  used  to  develop  each  of  the 
components  of  a  lesson  module.  The  pretest  is  used  to  permit  a  high  or 
middle  aptitude  student  to  advance  to  the  next  lesson  if  he  demonstrates 
sufficient  prior  knowledge  of  the  subject.  The  lesson  review  items  are 
used  to  ascertain  if  the  desired  terminal  behavior  has  been  developed  by 
the  lesson  element.  The  instructional  programmer  follows  a  general  proce¬ 
dure  in  the  preparation  of  the  lesson  module.  First,  a  pretest  and  lesson 
review  items  a^e  prepared,  then  the  lesson  element  is  written.  Slides, 
training  devices,  audio  scripts  and  special  video  displays  are  originated 
concurrently  in  the  preparation  of  the  lesson  module.  Then,  the  module 
is  coded  for  computer  implementation.  This  is  a  separate  task,  and  may 
be  done  either  as  the  lesson  is  being  written  or  after  the  instructional 
content  is  completed. 


The  sample  model  is  similar  to  the  one  used  in  the  CAI  Project  at 
USASCS.  It  is  rather  straight-forward  and  uncomplicated,  but  it  must 
be  remembered,  the  more  complex  the  model,  the  greater  the  task  and 
time  required  to  prepare  the  lesson  material.  Variations  will  naturally 
be  required  for  special  lessons  such  as  the  conduct  of  practical  exercises. 

EVALUATION 

The  CTS  evaluation  division  will  conduct  Phase  V  of  the  general  develop¬ 
ment  plan  as  provided  for  in  the  CTS  Management  Plan.  As  indicated  in 
the  plan,  ”This  phase  will  be  conducted  by  the  Product  Manager  concurrently 
with  Phase  IV  (CTS  Operation)  and  is  concerned  with  the  feasibility  and 
effectiveness  of  the  entire  system.  Thus,  one  of  the  focal  targets  set  for 
this  CTS  Project  is  an  in-depth  and  in-breadth  evaluation  of  the  CTS 
prototype,  both  from  a  components  and  a  systems  point  of  view. 

From  the  start,  it  was  decided  that  the  CTS  system  should  not  be  viewed 
*'in  vacuo’'.  It  was  readily  apparent  that  the  CTS  Project  carried  with  it 
a  definite  evolutionary  history  and  had  certain  prospects  for  future  develop¬ 
ment  and  operational  use  as  well.  Thus,  it  is  to  be  expected  that  the  final 
CTS  evaluation  plan  will  be  a  resultant  of  several  vectors  including: 

(a)  past  impetus  of  CONARC/USASCS  CAI  efforts  (e.  g. ,  summative  evalu¬ 
ation/ Task  Group  Report);  (b)  current  CTS  planning  (e..g.^  management/ 
time  phase  plans);  and,  (c)  Army  needs /expectations  (e.  g. ,  potential 
operational  problems).  Taking  into  consideration  information  from  these 
three  sources,  it  is  anticipated  that  the  ultimate  evaluation  product  will 
not  be  a  sterile  "in  vitro"  assessment  of  CTS,  but  a  realistic  "in  vivo" 
appraisal  with  many  practical  implications. 

At  the  present  time,  the  CTS  evaluation  division  is  mapping  out  the  ev^-^^ua- 
tion  domain  limits.  Five  basic  sub-areas  have  been  identir.ed  for  the 
feasibility  and  effectiveness  evaluation: 

1.  Hardware 

2.  Software 

3.  Cost 

4.  Courseware 

5.  Student 
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For  the  sake  of  expediency^  and  by  the  logic  of  their  relationships, 
these  five  basic  areas  will  be  consolidated  into  three  broad  evaluation 
areas,  which  incidentally  coincide  with  three  organization  branches 
within  the  CTS  evaluation  division: 

1.  Systems  effectiveness 

2.  Cost  effectiveness 

3.  Educational  effectiveness 

The  CTS  system  will  be  examined  from  a  hardware  and  software  com¬ 
ponents  perspective,  and,  ultimately  from  a  systems  perspective.  More 
specifically,  the  essential  functions  which  must  be  supported  by  the  CTS 
are  as  follows: 

A.  Interactive  Processing 
Student  Mode 
Author  mode 
Administrative  mode 
Proctor  mode 

B.  Batch  processing 
Author  processing 
Proctor  processing 
Administrative  processing 

The  evaluation  will,  of  course,  address  the  feasibility  and  effectiveness 
of  all  these  functions. 

The  cost  of  the  CTS  system  is,  of  course,  not  a  trivial  matter.  A  multi¬ 
dimensional  assessment  of  CTS  costing  will  be  undertaken  which  will 
include: 

1.  Type  of  cost  (Capital/Developmental/Operating) 

2.  Purpose  of  cost  (Direct/lndirect/Noninstructional) 

3.  Area  of  cost  (CAI/CMl/ sub-areas) 
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It  is  anticipated  that  the  derived  cost  model  will  not  just  represent  mere 
cold  statistics,  but  be  related  to  and  suggestive  of  degrees  of  educational 
benefits  accmied  by  the  costing  dimensions. 

Lastly,  the  pre-eminent  concern  of  educational  effectiveness  will  be 
investigated.  This  comprises  both  the  courseware  and  the  student.  In 
keeping  with  the  in-depth  and  in-breadth  nature  of  the  evaluation,  a 
formative- summative  paradigm  will  be  employed  to  assess  the  educational 
effectiveness  of  the  CTS.  By  definition,  the  formative  evaluation  will 
address  the  internal  validity/reliability  of  the  CTS  instructional  material. 

In  contrast,  the  summative  evaluation  will  focus  on  the  integrity  and 
validity  of  the  training  system  in  comparison  with  other  external  criteria 
(COBET /Field  Radio  Mechanics /Circuit  Conditioning)  as  reflected  in 
student  performance,  attitudes  and  other  predefined  indices  of  reference. 

Past,  and  recent,  contacts  with  many  external  agencies  and  associations 
(ADIS,  ETS,  NYIT,  IBM,  etc. )  have  assured  us  that  our  evaluation  plans 
are  contemporaneous  with  the  state-of-the-art.  For  added  insurance  in 
this  regard,  plans  are  also  being  made  to  organize  three  teams  of  con¬ 
sultants  for  the  three  basic  evaluation  areas  discussed  above. 

CURRENT  CAI  PROJECTS 

Although  there  are  literally  hundreds  of  CAI  experiments  being  conducted 
throughout  the  country,  few  attempts  are  being  made  at  sophistication  as 
many  potential  users  await  the  outcome  of  developments  being  sponsored 
by  the  National  Science  Fo\indation  and  by  the  Armed  Forces.  Most  new 
systems  use  teletype  compatible  terminals  in  the  simple  drill  and  practice 
mode.  The  trend  in  public  schools  is  to  take  advantage  of  the  consortium 
approach.  Meantime,  the  real  pioneers  continue  to  experiment  in  a 
variety  of  ways.  These  include  Suppes  at  Stanford,  Mitzel  at  Penn  State, 
Hansen  at  Florida  State,  Stolurow  at  Stonybrook,  Morgan  at  Montgomery 
County  (Md),  Bunderson  at  Brigham  Young,  and  Zinn  at  Wisconsin. 

The  largest  and  most  advanced  developmental  project  is  being  sponsored 
by  NSF  at  the  University  of  Illinois  under  the  direction  of  Dr.  Bitzer, 
another  of  the  pioneers.  The  system,  known  as  PLATO  IV,  utilizes  a 
large  scale  computer  and  is  being  designed  to  support  as  many  as  4000 
terminals  provided  access  to  no  more  than  400  different  subject  areas  is 
required  at  the  same  time.  The  terminal  consists  of  a  relatively  low-cost 
plasma  tube  display  with  a  keyboard,  a  touch  panel  and  microfiche  projection. 
These  terminals  will  eventually  be  located  on  campuses,  in  schools,  and 
on  military  installations  for  test  purposes.  The  three  Armed  Forces  are 
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participating  through  sponsorship  of  the  Advanced  Research  Planning 
Agency  (ARPA).  The  entire  project  will  be  evaluated  by  ETS  for  the 
next  three  years. 

The  second  NSF  sponsored  project  is  being  conducted  under  a  contract 
with  MITRE  Corporation.  The  contract  calls  for  experimental  courses 
in  reading  and  math  at  two  junior  colleges,  one  in  Virginia,  and  the 
other  in  Arizona.  The  system  uses  the  mini- computer  concept  to 
support  up  to  128  student  terminals.  The  terminal  consists  of  a  modi¬ 
fied  commercial  television  set  and  a  keyboard.  Random  access  sound 
is  still  questionable.  Course  material  for  use  on  this  system  is  being 
prepared  at  Brigham  Yoxing  under  Or.  Bunderson.  Evaluation  again  will 
be  by  ETS. 

The  United  States  Air  Force  has  prepared  specifications  and  requested 
bid  on  a  system  sinnilar  to  the  PLATO  IV.  Schedule  calls  for  develop¬ 
ment  and  evaluation  over  a  four  year  period. 

The  United  States  Navy  is  concentrating  on  Computer  Managed  Instruction 
(CMI).  A  large  scale  experiment  using  the  computer  to  direct  students 
through  courses  and  maintain  records  of  their  progress  is  currently 
being  conducted  at  the  training  center  in  Memphis,  Tennessee. 

From  the  above,  it  can  be  seen  that  results  of  all  large  experiments 
are  due  during  the  1975  -  1977  time  frame.  It  is  unlikely  that  any  other 
major  developments  can  be  completed  and  tested  prior  to  that  time. 

SUMMARY 

Project  ABACUS  has  as  its  purpose  the  development  and  evaluation  of 
a  large  scale  prototype  Computerized  Training  System  (CTS),  which 
combines  CAI  and  CMI  functions.  The  goal  is  a  suitable,  low  cost, 
viable,  and  effective  hardware  system,  and  a  newly  developed  language 
that  will  facilitate  course  development  and  provide  maximum  flexibility. 
From  the  prototype,  design  specifications  can  be  prepared  to  meet  the 
needs  of  CONARC  schools  and  training  centers. 
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OBSERVATIONS  AND  SUMMARY 

DR.  RICHARD  W.  BURNS 

Resource  Consultamt^  Soft-Skill  Training  Developments 


1.  The  workshop  was  well  attended  and  the  attendees  were  interested  as 
evidenced  by  the  large  number  of  questions  directed  to  each  speaker, 

2.  The  conference  format  was  tightly  scheduled  (I  know  why)  and  perhaps 
in  the  future  more  time  should  be  allowed  for  one-  to-  one  interaction 
between  attendees  and  speakers, 

3.  Your  personal  chairmanship  is  to  be  commended  for  keeping  to  the 
schedule. 

4.  One  theme  which  ran  through  most  of  the  presentations  involved  the 
difficulties  experienced  in  training  where  the  objectives  involved  people- 
people  types  of  performances.  Specifically  I*m  referring  to  skills  relating 
to  leader ship^  decision  making,  problem  solving,  communication  and  peer 
relationships.  1  was  pleased  to  note  that  much  progress  in  this  area  has 
already  been  made  and  that  continued  efforts  are  being  implemented. 
Further  exploration  of  role  playing,  simulation  and  trainee  involvement 

in  interactive  process  should  result  in  positive  benefits  to  the  training 
programs. 

5.  The  military  recognizes  and  is  concerned  about  the  changing  role  of 
the  military  system,  the  changing  role  and  quality  of  the  trainees  and  the 
changing  role  of  instructors  in  the  training  programs.  This  is  a  healthy 
point  of  view  as  evidenced  by  the  provisions  for  feedback  mechanisms  to 
the  system, 

6.  I  sense  that  the  task  of  training  instructors  is  made  more  burdensome 
because  of  the  high  rate  of  turnover,  I  would  recommend  that  this  problem 
be  thoroughly  studied  and  efforts  be  exerted  to  extend  the  Instructor  MOS 
to  a  greater  period  of  time. 

7.  Greater  emphasis  should  be  placed  on  retraining  instructors  for  those 
instances  in  which  their  roles  are  more  of  a  "resource  person"  than  the 
traditional  role  of  being  an  information  giver. 


(  ! 


V-126 


'1 

! 

i 


8.  Several  questions  arose  relating  to  how  an  instructional  (training) 
process  can  or  should  be  validated.  I  would  suggest  that  the  matter  of 
validation  of  performance  items  as  related  to  performance  objectives  are 
a  function  of  the  process  (the  degree  to  which  the  learner  has  interacted 
with  the  instructional  and  consequential  events)  and  only  "face  validity"  is 
of  concern.  This,  in  fact,  is  a  much  simpler  problem  than  the  validation 
of  achievement  tests  which  are  graded  (purpose  is  to  rank  the  learner). 

9.  Also,  several  questions  arose  dealing  with  the  topic  of  program  evalua¬ 
tion.  I  suggest  that  this  whole  area  is  one  of  great  weakness  in  most  any 
type  of  instructional  programs  and  that  in  fact  new  strategies  in  this  area 
need  to  be  developed.  However,  it  is  also  equally  obvious  to  me  that  in 
training  situations,  not  only  the  trainee  (as  a  graduate)  but  also  the  products 
of  the  trainee  must  be  evaluated.  For  example,  if  there  is  an  instructor 
training  school,  then  the  effectiveness  of  the  "instructor  training  program" 
should  be  measured  (evaluated)  by  a  follow  up  on  the  instructor  and  addition¬ 
ally,  the  instructors  products,  the  ones  whom  he  instructs.  This  concept 
is  related  to  the  general  concept  of  educational  accountability. 

10.  Program  evaluation  was  also  quite  frequently  mentioned  in  connection 
with  the  phrase  "cost-effectiveness".  I  did  not  perceive  that  cost-effective¬ 
ness  was  the  only  criterion,  exempting  cognitive  aclievement,  psychomotor 
achievement,  etc. ,  perceived  to  be  used  in  program  evaluation.  Nor,  am  I 
denying  the  importance  of  cost-effective  criteria.  However,  I  do  wish  to 
point  out  that  cost-effectiveness  is  not  the  only  criterion  and  that  when  used 
it  must  be  interpreted  extremely  carefully.  For  example,  if  information 
acquirement  were  to  be  an  objective  one  can  generally  demonstrate  that 
"drill"  as  a  process  is  more  cost-effective  than  "discovery  learning"  as  a 
process,  assuming  immediate  terminal  or  sximmative  evaluation.  In  the 
long  run,  delayed  summative  evaluation  may  show  that  "discoveiry  learning" 
is  the  most  cost-effective  considering  that  information  acquired  by  drill  is 
quickly  forgotten.  This  example  is  an  oversimplification  of  the  problem  but 
I  hope  has  been  clear  in  pointing  out  the  nature  of  the  problem  area  which  is 
obviously  not  merely  a  function  of  "information"  nor  "drill". 

11.  In  connection  with  Dr.  J.  P.  Fry’s  presentation,  an  important  question 
arose  which  time  did  not  permit  to  be  effectively  answered.  The  question 
dealt  with  how  feedback  to  the  instructional  system  could  be  effective  when 
the  system  was  one  of  peers  working  together  to  learn  and  where  peers  were 
also  involved  in  evaluation.  Might  I  suggest  that  in  discovery  situations, 
student  structured  situations  and  peer  evaluation  situations,  feedback  is 
provided  by  recording  in  some  fashion  the  elements  of  the  processes  can  be 
compared  to  terminal  performance  and  those  processes  providing  the  most 
effective  (time-cost-achievement-motivation)  interaction  as  measured  by 
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terminal  performance  can  be  singled  out  for  use  in  instructor  training.  In 
these  cases  the  instructors,  as  resource  persons  or  monitors,  are  aware 
of  and  orepared  to  direct  the  learning  process  in  proven,  meaningful  ways. 

12.  I  was  somewhat  disturbed  by  the  question  concerning  the  use  of  the  new 
Army  General  Classification  battery  scores  in  ways  other  than  intended. 

This  question  seemed  to  strike  a  sympathetic  note  with  other  attendees. 
Evidently  there  is,  in  the  minds  of  some  Army  personnel,  a  rather  widespread 
belief  in  and  a  desire  to  use  test  data  for  multiple  purposes.  This  is  the 
concept  of  the  "all  purpose  test".  I  would  like  to  point  out  that  tests  are 
constructed  for  specific  purposes  and  that  the  inclusion  of  test  data  on  record 
where  that  data  is  likely  to  be  misused  or  used  for  purposes  other  than 
intended  should  receive  the  attention  of  the  proper  authorities.  If  in  fact, 

the  Sergeant  Majors  School  wishes  diagnostic  information  relating  to  langxiagt 
arts  skills,  they  should  rely  on  data  specifically  gathered  for  that  purpose. 

13.  I  felt  that  the  presentations  of  this  conference  were  very  well  prepared 
and  well  received.  Additionally,  the  presentations  interfaced  unbelievably 
well  so  that  continuity  was  not  lost.  The  number  of  questions  asked  indicated 
a  great  deal  of  interest  in  soft- skill  type  of  problems. 

14.  Since  I  genuinely  appreciated  the  opportunity  to  participate  in  the  con¬ 
ference  I  must  express  an  opinion  which  I  hope  is  favorably  received.  I 
feel  that  others  (like  college  professor  types)  who  are  in  the  civilian  segment 
would  not  only  be  interested  in  attending  such  conferences  as  this  but  would 
profit  from  participating,  even  as  an  observer.  I  think  a  mutual  appreciation 
and  understanding  of  what  the  military  is  doing  in  the  area  of  teaching -training 
is  important,  not  only  to  us  (the  civilians)  but  also  to  the  Army.  You,  in  the 
military  are  farther  along  in  applications  of  technology,  systems,  hardware, 
software,  objectives,  etc. ,  (the  whole  bit)  than  are  the  public  schools, 
colleges  and  Universities. 

15.  Finally,  I  detected  a  feeling  from  conversations  with  attendees  that  the 
schools  (military)  do  not  always  share  what  they  are  doing.  Perhaps  this  is 
past  history,  but  if  not,  then  more  conferences  similar  to  this  one  should  be 
held  on  a  regular  basis.  More  learning  and  greater  progress  will  result 
from  seeing  what  ones  peers  or  counterparts  are  doing  than  will  result  from 
any  number  of  directives,  injunctions  and  orders.  I  believe  I  might  say  this 
another  way;  specifically  that  the  attendees  were  sometimes  at  a  loss  as  to 
what  to  do  until  shown  by  someone  else.  They  did  profit  from  this  conference 
and  they  did  get  new  ideas. 
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